cURL HTTP1.1 empty POST bug

Today we spent almost 2 hours on a weird discrepancy between our development and staging environments. It’s pretty rare, that I experience such low level issue, thus in my opinion it’s worth mentioning here.

Last weeks we were busy developing integration to a new data vendor. Everything went well until we deployed the application to stage.
Suddenly we started receiving HTTP status 411 on one of the calls. Since we work with cURL library, which we believed is stable enough, we thought the problem is somewhere between the source code and environment configuration.

Later we found that the same request gets accepted if sent from a client other than cURL (e.g. chrome-poster). The unique about this request is that it’s sent with POST method (the vendor’s strict requirement) but the content body is empty.

In the end we discovered that newer version (since 7.20) of cURL interprets missing body as a negotiation request – sends Expect: 100-continue header and Content-Length: -1.

So, the immediate solution was to send empty content body (zero-length string) to cURL, which aligned the behavior in all the environments.

On the way we discovered a useful option CURLINFO_HEADER_OUT, which enables a possibility to further retrieve the headers sent by cURL to the remote host. From now we use it in our error handling mechanism to trace the sent headers as well.

What can we conclude from this story?

1. Try to synchronize the software of all of your environments. If possible, use exactly the same version OS version, libraries, tools, etc. It’s very easy if you host your applications on VPS and use VMs for development and staging servers. The least convenient case is when you have OSs of different architecture and (e.g. Windows for development and Linux for production).
2. Don’t underestimate importance of error handling. Find the optimal level of handling for your application, which will be easily extended and configured.

Hope my sincere advices will help you, my dear friend, to save your precious time.

FreeBSD MySQL MyISAM multi-CPU concurrency performance

Recently we had a performance issue with multiple plain concurrent requests on a MyISAM table – the process list reported that all selects get stuck for very long time while Sending Data. After many tests we discovered that:

1. The problem only occurs when running on multiple CPUs;
2. The problem only occurs when running on FreeBSD OS.

After searching over the Internet we found, that this class of problems is being reported since antique versions of FreeBSD and, despite the claims it’s fixed with each version, it persists even in fresh 8.0.

So the decision was to install Linux on all the DB servers.

There is an alternative option though – it’s possible on FreeBSD to install MySQL of any version with WITH_LINUXTHREADS=yes parameter and it will be benefited from the Linux thread model.

Version Upgrade

Today I found that the new version 7.10 of Kubuntu just released.

Since my only Linux machine is in the office and I’m at home today, I’ve started to upgrade the system via VNC. And now I’m wondering will it allow me also to finish with it. 🙂

Update: It’s funny, but it didn’t allow me to continue, since some of my old Third-Party Feisty repositories reported 404. Continuing the jorney…

Final: Well, the upgrade succeeded from the 3rd attempt. The system seems working a bit faster, probably because of KDE major upgrade.