*** glibc detected *** free(): invalid next size (fast): 0x083de008 ***

One of the very frustrating problem I have seen ever. We first observed this problem in an application of ours on one of our test machine which is a RHEL4 using glibc 2.3.x.

From the error message it is pretty much clear that the problem is because of freeing some invalid pointer. Since our application is a multi-threaded application, an obvious fix was to protect the area where the deletion is happening. We did that. But, the fix didn’t solve the issue but delayed it by 8 hours.

The variable that is being deleted is a char array which has been allocated memory using new operator. We copy the contents in to it using memcpy. Since the new operator handles memory allocation properly, we are not  very worried about the contents being copied. If the app is trying to copy some thing out of the boundary, it is expected that it would crash there. So we continued looking ahead and tried to lock the areas where the variable got created and initialized.  This didn’t solve the problem. but the problem got delayed for 16 days :)..

Yes!! Now comes the major problem… How the hell would you be able to solve an issue that happens only after 16 days run?

We went ahead and looked at all the possibilities. Couldn’t find a clue about what might be gone wrong.

For some reason I got a doubt that the memcpy is trying to copy something out of the bounds. I had this nagging feeling and so I went ahead and came up with a small test sample that replicates the code where we are seeing the crash. It gave us surprising results…

Here is the code that i have used….

#include <stdio.h>
#include <string.h>
int main()
{

char *temp;

int len=12;

temp = new char[len]; // Allocate 12 bytes of memory

printf(“Got the new object\nAllocating the memory now\n”);
temp[0]=’A’; // used the first
printf(“Allocated size =1\n”);
memcpy(temp+1,”BBBB”,4); // used 5 bytes
printf(“Allocated size =5\n”);
memcpy(temp+5,”CCCC”,4);//using 9 bytes
printf(“Allocated size =9\n”);
memcpy(temp+9,”DDDD”,4);// using 13 bytes… I expected to crash here.. But it didnt 😦
printf(“Allocated size =13\n”);
memcpy(temp+13,”EEEE”,4);// using 17 bytes… No crash either
printf(“Allocated size =17\n”);
printf(“%s”,temp); // Printing all the 17 bytes…  Yes it does

delete[] temp; // deleting the variable
return 1;
}

I ran this piece of code on SUSE linux with glibc 2.2 and it didnt core. I saw the same results in HPUX, Solaris and AIX machines. No crashes either at memcpy or at delete. That was very surprising for me. I went ahead and ran the test program on RHEL 4 box.

Bingoo!! It cored and it cored with same errror. The machine has glibc2.3

*** glibc detected *** free(): invalid next size (fast): 0x083de008 ***

Initially I thought it depends on might depend on glibc version as the SUSE linux I had was using glibc2.2.

I tried running the test on all the machines where i could lay my hand on and the test results werent conclusive.

Most surprising thing was when i ran the tests on two RHEL5 machines with glibc 2.5.34. One of them cored with same error and other didn’t. I am still not sure the exact reason for this issue. But I made a few changes in my code to ensure that we dont copy out of the bounds.. some thing like this…

if(sizeof(temp1)> k){
memcpy(temp1,temp2,k);}
else{
return fail;}

This solved our issue. I hope this helps

Advertisements