cURL HTTP1.1 empty POST bug
Today we spent almost 2 hours on a weird discrepancy between our development and staging environments. It’s pretty rare, that I experience such low level issue, thus in my opinion it’s worth mentioning here.
Last weeks we were busy developing integration to a new data vendor. Everything went well until we deployed the application to stage.
Suddenly we started receiving HTTP status 411 on one of the calls. Since we work with cURL library, which we believed is stable enough, we thought the problem is somewhere between the source code and environment configuration.
Later we found that the same request gets accepted if sent from a client other than cURL (e.g. chrome-poster). The unique about this request is that it’s sent with POST method (the vendor’s strict requirement) but the content body is empty.
So, the immediate solution was to send empty content body (zero-length string) to cURL, which aligned the behavior in all the environments.
On the way we discovered a useful option CURLINFO_HEADER_OUT, which enables a possibility to further retrieve the headers sent by cURL to the remote host. From now we use it in our error handling mechanism to trace the sent headers as well.
What can we conclude from this story?
- Try to synchronize the software of all of your environments. If possible, use exactly the same version OS version, libraries, tools, etc. It’s very easy if you host your applications on VPS and use VMs for development and staging servers. The least convenient case is when you have OSs of different architecture and (e.g. Windows for development and Linux for production).
- Don’t underestimate importance of error handling. Find the optimal level of handling for your application, which will be easily extended and configured.