Let’s say we have an application where all tests run fine. Now you make a change which breaks the tests, you commit it to the repository, and you go on to work on other stuff. Later, I run the tests and see that they fail. What should I do? Should I be the “good” guy and fix them myself and provide a patch for you? Or should I be the “bad” guy that simply says: “Hey, you broke the tests, please fix them”? The first approach teaches you it’s ok to be lazy, there is someone else who cleans up, whereas the second approach hopefully teaches you to run the tests the next time before committing code ;-)

@nate: Well, maybe my test environment is not set up correctly. That’s what I did to make the tests run: I changed the value of CAKE_CORE_INCLUDE_PATH in test.php (I’m using an advanced installation) and defined the $test database connection. I don’t know whether there are additional steps necessary. The fatal error I get is because I don’t have the xdebug extension installed, see my ticket.

I agree with you, that I’m not always an “easy” guy to work with. I’m very analytical and critical, and not everyone can deal with that. In the past I even got fired because I gave my boss negative feedback ;-) And I’m probably not always that diplomatic.

Regarding “… work [..] in a constructive way”, I think we have different definitions of what that means. For example: I opened the tickets to encourage you to fix them (as I think it is bad to have failing tests) whereas from your point of view they were useless.

Anyway, thanks for your feedback!

@Jonathan: The respective tickets are tickets #4596-#4600. In those tickets I simply reported the failing tests, as I didn’t knew whether the problems were in the test cases, in the tested classes, or in the new test suite shell.

I just ran all the tests, and had others run them as well. There are a very few failing tests, on code which is still in-development (as you say, it is part of the process). There are no fatal errors.

Also, none of the tests that currently fail were mentioned in the tickets you opened, and none of the tickets you opened have any failing tests (when run from the web test interface).

This very debate illustrates the crux of my issue with you: you’re an intelligent programmer, and we could always, always use more people like you to help improve the Cake core, but in my experience you’re not able to work with other people in a constructive way. Your main interest always seems to be in blaming others while doing everything possible to avoid blame yourself.

I hope that in the future you’re able to prove me wrong.

IMO submitting patches is teaching precisely how to fish. Providing patches is the cornerstone of OSS development. If every bug in every OSS was left for the developer to fix we would not be here and so far out.

To paraphrase Linus “Talk is cheap – Show me the code” is universally true.

@Mariano: No.

@Brendon: Sure, it is a point of view thing, and you can see it like you described.

@nate: Well, I just set up the “normal” test environment and ran the tests, and the same tests fail again (plus some other tests), even the test suite fails with a fatal error ;-)

@Tarique: I think taking the stance “the person who breaks it [the tests] fixes it” is independent of whether I’m a team member or not. Sure, I may be able to supply patches, but it is not satisfying to clean up the “mess” of others only because they are too lazy to do it themselves. Somehow it would be like giving them a fish instead of teaching them how to fish…

@Daniel – when you are in the team taking a stance “the person who breaks it fixes it” is correct but from if you are in the community then not supplying patches when it is well known that you *can* will be considered “whining”

@Mariano – That was as rhetorical question right? :P

You make some excellent points about the cost of failing tests and the importance of testing across different environments. However, you seem to be mischaracterizing the situation in this case.

The particular tests to which you’re referring aren’t actually broken, in fact, they all pass just fine. However, those tests were written to run in the context of an HTTP request, and you’re running them on the command line. This isn’t just a different “environment,” this is an entirely different context.

Because of this, and because of the fact that the new test suite shell command is as-yet-unreleased, “fixing” those tests (i.e. setting up the necessary environmental conditions to run them on the CLI) hasn’t been a high priority, because as I mentioned, nothing is actually broken.

Therefore, your idea about the “costs” doesn’t really apply, since the necessary environmental bootstrapping is a fixed cost, and will apply the same across the board to all affected tests.