Double free or corruption crash using cutscenes

Where all the bugs are reported and discussed
Post Reply
virtual
Posts: 15
Joined: Sat Jan 20, 2018 11:52 am

Double free or corruption crash using cutscenes

Post by virtual » Mon Jan 22, 2018 10:19 pm

Hi, I'm trying to use Comskip compiled from the latest git in a Debian 9.3 x86_64 VM.
I built Comskip in this way:

Code: Select all

apt-get install -y git build-essential libargtable2-dev libsdl1.2-dev libavformat-dev autoconf libtool
git clone git://github.com/erikkaashoek/Comskip
cd Comskip
./autogen.sh
./configure
make
and it appears to run successfully, but when I pass a cutscene dump file (created from the Windows comskip binary) it crashes.
I'm unable to capture the output directly because it prints all the libraries like below (and output redirection doesn't help):

Image

So I've tried running the same command under strace and the output is available at this gist:

Code: Select all

https://gist.github.com/virtualdj/bc09f4073769924b52761e76e71ecede#file-gistfile1-txt-L916-L928
It crashes after reading the cutscene file "./paramount1.dmp" (which is in the same directory than "comskip" and "paramount.ini"), on line 928.

The strange thing is that if I run the same command line under "valgrind" instead of "strace" it doesn't crash and it creates the output correctly! Is this a bug?

virtual
Posts: 15
Joined: Sat Jan 20, 2018 11:52 am

Re: Double free or corruption crash using cutscenes

Post by virtual » Sat Jan 27, 2018 3:58 pm

Is there anything I can do to provide more information to solve this issue?
I really would like to run Comskip in Linux and then move it into my NAS (which has the same crashing issues with cutscene, this is why I'm trying debugging in a clean Debian VM).

EDIT:
Even by setting "verbose=10" on the INI file, the content of the log is a laconic:

Code: Select all

Loaded 24325 bytes from cutfile "paramount1.dmp"
and nothing else. Doesn't seem a permission issue, as I'm running Comskip as root.

erik
Site Admin
Posts: 3297
Joined: Sun Aug 21, 2005 3:49 pm

Re: Double free or corruption crash using cutscenes

Post by erik » Wed Feb 07, 2018 7:31 am

Thanks, I see there is a warning on memory corruption or a double free.
Helps to find the problem.

erik
Site Admin
Posts: 3297
Joined: Sun Aug 21, 2005 3:49 pm

Re: Double free or corruption crash using cutscenes

Post by erik » Wed Feb 07, 2018 8:18 am

What is the size of the generated cutscene dump file?

virtual
Posts: 15
Joined: Sat Jan 20, 2018 11:52 am

Re: Double free or corruption crash using cutscenes

Post by virtual » Wed Feb 07, 2018 7:09 pm

erik wrote:Thanks, I see there is a warning on memory corruption or a double free.
Helps to find the problem.
Thanks!
erik wrote:What is the size of the generated cutscene dump file?
Actually I've generated the "paramount1.dmp" and "paramount2.dmp" on a different machine (using ComSkip GUI for Windows) and then I'm moving the files to a headless Debian VM which crashes. So basically I'm not "generating" anything on the VM that crashes.
And the same files work in the Windows version.

The size of each of the two dump files is exactly 24329 bytes.

erik
Site Admin
Posts: 3297
Joined: Sun Aug 21, 2005 3:49 pm

Re: Double free or corruption crash using cutscenes

Post by erik » Wed Feb 07, 2018 9:44 pm

Please test first with files generated on linux just to make sure the file format is the same
Does not have to be relevant dump file, just to test reading

virtual
Posts: 15
Joined: Sat Jan 20, 2018 11:52 am

Re: Double free or corruption crash using cutscenes

Post by virtual » Thu Feb 08, 2018 7:25 pm

OK, so I've created another subfolder test2 to test creating cutscenes directly on Linux VM.
I cannot use a GUI, so I hope the following is correct to "extract" cutscenes from the video file:

Code: Select all

root@debian9:~/cskip/Comskip/test/test2# ../comskip --dump=944 --output=. ../Sample.ts 
Comskip 0.81.098, made using ffmpeg
Donator build
The commandline used was:
../comskip --dump=944 --output=. ../Sample.ts

No INI file found in current directory.  Searching PATH...
INI file found at comskip.ini
No INI file found anywhere!!!!
Setting dump frame number to 944 as per command line.
 0:09:00 - 15498 frames in 24.79 sec(625.17 fps), 1.00 sec(632.00 fps), 99%
15534 frames decoded in 24.84 seconds (625.36 fps)
Commercials were found.

root@debian9:~/cskip/Comskip/test/test2# mv Sample.dmp sample1.dmp

root@debian9:~/cskip/Comskip/test/test2# ../comskip --dump=1359 --output=. ../Sample.ts 
Comskip 0.81.098, made using ffmpeg
Donator build
The commandline used was:
../comskip --dump=1359 --output=. ../Sample.ts

No INI file found in current directory.  Searching PATH...
INI file found at comskip.ini
No INI file found anywhere!!!!
Setting dump frame number to 1359 as per command line.
 0:08:36 - 14893 frames in 22.80 sec(653.20 fps), 1.00 sec(622.00 fps), 95%
15534 frames decoded in 23.71 seconds (655.17 fps)
Commercials were found.

root@debian9:~/cskip/Comskip/test/test2# mv Sample.dmp sample2.dmp
root@debian9:~/cskip/Comskip/test/test2# rm *.txt
root@debian9:~/cskip/Comskip/test/test2# ls -l
total 48
-rw-r--r-- 1 root root 24329 Feb  8 19:13 sample1.dmp
-rw-r--r-- 1 root root 24329 Feb  8 19:14 sample2.dmp

root@debian9:~/cskip/Comskip/test/test2# md5sum sample*.dmp
b1928bec7009573ebebcbd46d31b4181  sample1.dmp
b1928bec7009573ebebcbd46d31b4181  sample2.dmp
Two questions here:
  1. Is it correct that comskip has to read the whole video just to extract a single frame? I deliberately supplied a missing INI file just to extract the cutscene without scanning, is this correct or is there a faster solution? Can I extract multiple frame numbers with one single command line?
  2. Why even if I wrote 2 different frame numbers the content of the two DMP files (actually the hash) is the same? The 2 dmp files that I extracted on Windows are actually different, though.
Then I copied my INI file and changed the filenames and, as you can see, it crashes.

Code: Select all

root@debian9:~/cskip/Comskip/test/test2# cp ../paramount.ini sample.ini

< edit with nano >

root@debian9:~/cskip/Comskip/test/test2# grep "cutscenefile" sample.ini 
cutscenefile1="./sample1.dmp"
cutscenefile2="./sample2.dmp"
cutscenefile3=""
cutscenefile4=""
cutscenefile5=""
cutscenefile6=""
cutscenefile7=""
cutscenefile8=""

root@debian9:~/cskip/Comskip/test/test2# ../comskip --ini=./sample.ini --output=. ../Sample.ts
Comskip 0.81.098, made using ffmpeg
Donator build
The commandline used was:
../comskip --ini=./sample.ini --output=. ../Sample.ts

Setting ini file to ./sample.ini as per commandline
Using ./sample.ini for initiation values.
Loaded 24325 bytes from cutfile "./sample1.dmp"
*** Error in `../comskip': double free or corruption (top): 0x000055b5038e88b0 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x70bcb)[0x7fd42df5fbcb]
/lib/x86_64-linux-gnu/libc.so.6(+0x76f96)[0x7fd42df65f96]
/lib/x86_64-linux-gnu/libc.so.6(+0x777de)[0x7fd42df667de]
/lib/x86_64-linux-gnu/libc.so.6(fclose+0x132)[0x7fd42df55c12]
../comskip(+0x12ae4)[0x55b4ff280ae4]
../comskip(+0x2c2d5)[0x55b4ff29a2d5]
../comskip(+0x3863)[0x55b4ff271863]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1)[0x7fd42df0f2b1]
../comskip(+0x433a)[0x55b4ff27233a]
I hope this helps!

erik
Site Admin
Posts: 3297
Joined: Sun Aug 21, 2005 3:49 pm

Re: Double free or corruption crash using cutscenes

Post by erik » Thu Feb 08, 2018 7:35 pm

It helps to exclude certain errors.
Indeed you have to scan the whole file for a single cutscene to be created, this because seeking is not reliable on certain recordings

virtual
Posts: 15
Joined: Sat Jan 20, 2018 11:52 am

Re: Double free or corruption crash using cutscenes

Post by virtual » Thu Feb 08, 2018 8:20 pm

OK, that's fair. I hope I'll be able to move the DMP file from Windows to Linux, though.
If you need something else to track the crash, let me know.

virtual
Posts: 15
Joined: Sat Jan 20, 2018 11:52 am

Re: Double free or corruption crash using cutscenes

Post by virtual » Thu Feb 08, 2018 9:34 pm

I tried running under GDB and this is the result (unfortunately I'm not keen on C/C++):

Code: Select all

root@debian9:~/cskip/Comskip/test/test2# gdb --args ../comskip --ini=./sample.ini --output=. ../Sample.ts 
GNU gdb (Debian 7.12-6) 7.12.0.20161007-git
... cut ...
Reading symbols from ../comskip...done.

(gdb) run
.. cut ..
Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.

(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007ffff56b53fa in __GI_abort () at abort.c:89
#2  0x00007ffff56f1bd0 in __libc_message (do_abort=do_abort@entry=2, 
    fmt=fmt@entry=0x7ffff57e6dd0 "*** Error in `%s': %s: 0x%s ***\n")
    at ../sysdeps/posix/libc_fatal.c:175
#3  0x00007ffff56f7f96 in malloc_printerr (action=3, 
    str=0x7ffff57e6f08 "double free or corruption (top)", ptr=<optimized out>, 
    ar_ptr=<optimized out>) at malloc.c:5049
#4  0x00007ffff56f87de in _int_free (av=0x7ffff5a1ab00 <main_arena>, 
    p=0x5555580498d0, have_lock=0) at malloc.c:3905
#5  0x00007ffff56e7c12 in _IO_new_fclose (fp=0x5555580498e0) at iofclose.c:84
#6  0x0000555555563dfa in LoadCutScene (
    filename=filename@entry=0x555557f05780 <cutscenefile1> "sample1.dmp")
    at comskip.c:9961
#7  0x0000555555566ae4 in LoadIniFile () at comskip.c:8591
#8  0x00005555555802d5 in LoadSettings (argc=<optimized out>, 
    argv=0x7fffffffebb8) at comskip.c:9068
#9  0x0000555555557863 in main (argc=4, argv=0x7fffffffebb8) at mpeg2dec.c:2174

erik
Site Admin
Posts: 3297
Joined: Sun Aug 21, 2005 3:49 pm

Re: Double free or corruption crash using cutscenes

Post by erik » Fri Feb 09, 2018 8:24 am

Many thanks for helping me.
I think I found and solved the problem.
The updated source is on github so if you can fetch the latest comskip version and rebuild and test?

virtual
Posts: 15
Joined: Sat Jan 20, 2018 11:52 am

Re: Double free or corruption crash using cutscenes

Post by virtual » Fri Feb 09, 2018 8:07 pm

erik wrote:Many thanks for helping me.
I think I found and solved the problem.
The updated source is on github so if you can fetch the latest comskip version and rebuild and test?
Thanks to you! It works perfectly now, also with the DMP files moved from Windows:

Code: Select all

root@debian9:~/cskip/Comskip/test/test2# ../comskip --ini=./sample.ini --output=. ../Sample.ts 
Comskip 0.81.098, made using ffmpeg
Donator build
The commandline used was:
../comskip --ini=./sample.ini --output=. ../Sample.ts

Setting ini file to ./sample.ini as per commandline
Using ./sample.ini for initiation values.
Loaded 24325 bytes from cutfile "sample1.dmp"
Loaded 24325 bytes from cutfile "sample2.dmp"
Mpeg:   ../Sample.ts
Exe     comskip.exe
Logo:   ./Sample.logo.txt
Ini:    ./sample.ini

Detection Methods to be used:
        1) CutScenes


ComSkip throttles back from -0001 to -0001.
The time is now 1833 so it's full speed ahead
... cut ...
----------------------------------------------------
Block list after weighing
----------------------------------------------------
  #     sbf  bs  be     fs     fe        ts        te       len     sc   scr cmb   ar                   cut    bri logo   vol sil   corr stdev   cc
  0:+     0   0   0      1  13559     0.00s   542.28s   542.28s   1.00  0.16   0 0.00  7      {       t    br 12780 0.00 10900 820  0.000   127           

13559 Frames Processed
Commercials were not found.

virtual
Posts: 15
Joined: Sat Jan 20, 2018 11:52 am

Re: Double free or corruption crash using cutscenes

Post by virtual » Sat Feb 10, 2018 9:47 am

Another small thing... I noticed when compiling the latest git version that it still displays version 0.81.098 when running the program, while you actually changed the version to 0.82.005 in comskip.h.
So I've proposed a PR in GitHub to automatically sync the version number.

Post Reply