While working on a recent pull request, I discovered a number of things that I thought I should draw attention to.
The changes I was making required that I rerecord the VCR cassettes for the associated spec tests. After rerecording the cassettes and ensuring the tests passed on my local branch, I pushed my changes to the remote, only to have the tests fail when run through Travis. I found this puzzling, because my remote and local branches were in sync and the tests passed on my local branch.
After further investigation, I determined that the global setting for “:allow_unused_http_interactions” is different when run on Travis vs locally - it’s false in Travis and true when run locally. Ah! so that’s why it’s not failing when I run it locally! So, I set it to false, run it locally, and sure enough it fails. Now I know why the behavior was different, but I don’t know why it’s failing.
Now that I can duplicate the failure, I try rerecording the cassettes again. Lo and behold, It fails while trying to rerecord the cassettes! “But how can you have unused HTTP interactions when you’re recording the interactions?” you may ask. “Good question!” I’d reply. But then I realized the tests were being looped through twice.
The first time through it records the interactions, and the second time it reads from the recording. Hmmm, the second time through, the same code is making fewer requests - there must be some caching going on. Sure enough, caching in the model object was the culprit. Clearing the cache between iterations resolved the problem.
So, the moral of the story is:
- Don’t assume Travis runs tests with the same settings as when you run tests locally.
- Beware of caching in VCR based tests that iterate more than once.