Seva's notes

Wisdom, Awareness, Responsibility :)

cURL HTTP1.1 empty POST bug

with 2 comments

Today we spent almost 2 hours on a weird discrepancy between our development and staging environments. It’s pretty rare, that I experience such low level issue, thus in my opinion it’s worth mentioning here.

Last weeks we were busy developing integration to a new data vendor. Everything went well until we deployed the application to stage.
Suddenly we started receiving HTTP status 411 on one of the calls. Since we work with cURL library, which we believed is stable enough, we thought the problem is somewhere between the source code and environment configuration.

Later we found that the same request gets accepted if sent from a client other than cURL (e.g. chrome-poster). The unique about this request is that it’s sent with POST method (the vendor’s strict requirement) but the content body is empty.

In the end we discovered that newer version (since 7.20) of cURL interprets missing body as a negotiation request – sends Expect: 100-continue header and Content-Length: -1.

So, the immediate solution was to send empty content body (zero-length string) to cURL, which aligned the behavior in all the environments.

On the way we discovered a useful option CURLINFO_HEADER_OUT, which enables a possibility to further retrieve the headers sent by cURL to the remote host. From now we use it in our error handling mechanism to trace the sent headers as well.

What can we conclude from this story?

  1. Try to synchronize the software of all of your environments. If possible, use exactly the same version OS version, libraries, tools, etc. It’s very easy if you host your applications on VPS and use VMs for development and staging servers. The least convenient case is when you have OSs of different architecture and (e.g. Windows for development and Linux for production).
  2. Don’t underestimate importance of error handling. Find the optimal level of handling for your application, which will be easily extended and configured.
Hope my sincere advices will help you, my dear friend, to save your precious time.

Written by Seva

2011-08-03 (August 3) at 12:47:35

2 Responses

Subscribe to comments with RSS.

  1. […] cURL HTTP1.1 empty POST bug […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: