Why you should test your CakePHP applications
My comment about me not trusting CakePHP caused quite some discussions. Some interpreted it in the way that I think CakePHP is crap (well, if that would be the case I wouldn’t use it *g*). Others criticized me for publicly mentioning that some tests fail in the development branch… Anyway, what I missed was a discussion about the implications of my statement. So I will catch up on that with this article.
One point I mentioned in the aforementioned comment are the missing tests. Not every class is tested. That’s because CakePHP was not developed with testing in mind originally. The first tests were added at the beginning of this year I think. And since then more and more tests have been written. But still, it will take quite a while to get a high test coverage. So the consequence of this is that when a developer makes changes in an untested class it is possible he will break something by accident that worked before, and he won’t notice it (you can find many such examples on trac). That means a working feature in version x may no longer work in version x+y.
The other point are the failing core tests in the development branch. They signal: “Hey, something doesn’t work”. Maybe it is “only” a broken test, maybe it is a broken feature, whatever. It doesn’t matter, the message is the same: something is broken. Sure, it can always happen that you commit something that breaks the tests, nobody is perfect. But if the tests fail over a longer time period, well, then the impression I get is that the developers don’t care about the tests.
Ok, I am now aware of those “trust problems”. But what can I do? It is no fun to write tests for existing, untested stuff, so writing core tests is not an option. And I cannot change the attitude of other developers towards tests from one day to the other. But I can write tests for my own application, and so test the underlying framework indirectly. If I update the framework, and my tests fail afterwards, then it is very probable they failed due to a newly introduced bug in the framework, and I can report it.
Of course, the aforementioned points are not the only motivation for me to write tests, the article “Top 12 Reasons to Write Unit Tests” lists other points that are important to me.
So my recommendation to everyone using CakePHP (or writing software in general): write tests, tests, tests!
Is there a tutorial knocking around about testing in CakePHP?
Tests really seem to make sense … as I am very new to this topic, I think I should have a look at the “cake/tests” directory
Regards, Christoph
@Neil, preloader: Thanks for your comments!
@Neil: There isn’t much documentation available about testing in CakePHP. You may try http://bakery.cakephp.org/articles/view/testing-models-with-cakephp-1-2-test-suite and look at the existing tests in cake/tests. I also wrote about this topic earlier (http://cakebaker.42dh.com/tags/testing), but most of the articles don’t deal with the testing infrastructure available in CakePHP.
@preloader: Yes, you can learn quite a bit from the existing tests.
Since vendor-branching cakephp I have been paying close attention to the 1.2 commits. I will say, that some of them leave me scratching my head:
rev 5636: “Fixing habtm associations”
My Thinking: “Well, damn. What was wrong with em
Sorry, post cut-off…
rev 5636: “Fixing habtm associations”
My Thinking: Well, damn. What was wrong with them before rev5636?
I agree with you 100% to “protect” yourself (and the vitality of your app) by creating a series of local (app specific) tests. But, the question that I’ve always had has been: What do I test and how far do I take it? Do I do my own core lib tests or just test within the constraints of my app?
@all
Why doesn’t cake have more bug-fix releases? There has been over 200 commits (and 2 months) since the last release which most of them address fixes for trac tickets.
@devs:
It would help so much with testing these branches if you could be more helpful in conveying to us what these commits include. For instance…
rev9998
“Bug Fix (Critical) : Fixed something in _file_ that prevented it from working correctly. This revision should be patched immediately.”
That would make my day! I could have a simple program to check the logs for these keywords (such as “Critical”).
@Brandon: Thanks for your comment.
> What do I test and how far do I take it?
Good question. I usually test model functions, helpers and components. And depending on the project I also test the workflow with Selenium.
> Do I do my own core lib tests or just test within the constraints of my app?
Personally, I don’t write core lib tests as it is boring for me ;-)
Yeah I agree with you that it would be useful if there are more bug-fix releases or at least regular releases, e.g. every six weeks a release.
@Brandon
Cake is released instantaneously:
svn co https://svn.cakephp.org/repo/branches/1.2.x.x 1.2.x.x_latest
If that breaks something, you can also move backwards with:
svn co –revision #### https://svn.cakephp.org/repo/branches/1.2.x.x 1.2.x.x_latest
to find out which check in caused the problem and either fix it in your app or write a test/raise a ticket (replace #s with the revision you want).
https://trac.cakephp.org/timeline is the page to see what’s being done and:
https://trac.cakephp.org/report/1 will give you all the info you need to track critical stuff.
It is, after all, still alpha software (but still able to be more stable than some proprietary stuff I’ve used).
~GreyCells
No mather how much I try, I still can’t understand test. I guess that’s why people go to coledge. :)
@GreyCells: Thanks for this explanation!
@Proleter: What do you not understand?
What they do? i.e. How they test?
@Proleter: I am not sure I understand your question correctly.
Let me show you a simple example of a test. Here the class we want to test:
class HelloWorld { function getHello() { return 'hello world'; } }And here the test case where you define, what you expect the aforementioned class should do:
class HelloWorldTest extends CakeTestCase { function testGetHello() { $hello = new HelloWorld(); $this->assertEqual('hello world', $hello->getHello()); } }This test is then run via the test suite.
Is it now more clear?
@GreyCells
First, SVN commits are *not* releases! They could only be considered daily builds at best.
The bigger picture I want to paint is that a lot of developers (both new and old) feel safe with official releases. There are so many bug fixes that should be released in subsequent *maintenance* releases. I don’t understand why they are all held for these minor releases ever X months. I stand by the creed to “Release early, release often”… and it’s something that I feel the cake devs don’t follow (with possible good reason).
I think it’s well known that the 1.2 branch is alpha but you sure don’t see anyone saying “Don’t use it! Use 1.x instead!”. It’s very confusing for newcomers to cake when they hear so much hype and stability of the 1.2 branch then are hit with the “sorry, its alpha” scapegoat of an excuse when something breaks/changes.
Also, as users of cake we shouldn’t have to police these commits. It seems it’s commonly left up to us to decide which commits are safe. I think with some *BASIC* keywords in commit messages could help a ton — but that’s just me :)
The trac timeline should be renamed to “activity log” :). It doesn’t really help at all when it comes to seeing when stuff WILL be completed or addressed. It’s reactive-based (meaning it only shows what has been done).
I am not trying to complain, I just see that addressing these issues would benefit the development process.
Thank you @cakebaker. Yes it is…