Blog Just another tech site

25Jun/128

ZoneMinder + Foscam FI8918W. WAR Corrupt JPEG data: extraneous bytes before marker 0xd9

Foscam Extranenous bytes before marker error.

From my experiences the Foscam FI8918W does seem like a very solid, capable remote PTZ IP camera. The issues with it arose once it was linked in to a ZoneMinder system. It seemed for every frame that ZoneMinder received / analysed from the Foscam camera, it would produce an error message in the system log, quickly giving me thousands of entries all repeating the same exact warning.

zmc_m1[14935]: WAR [Corrupt JPEG data: 1 extraneous bytes before marker 0xd9]

After a quick search online I have found that it is a commonly occuring issue with many users online, all with ZoneMinder and the Foscam cameras. It seems that 'libjpeg' that is utilized for processing by ZoneMinder believes that the frames received are errorness / the structure is wrong and warns about it. It is only a warning and should not affect the actual workings of ZoneMinder. A thread about it on the Foscam forum can be found for information:

http://foscam.us/forum/fi8918wcorruptjpegdata-1extraneous-bytes-before-marker-0xd9-t1067.html

It seems the way most people have dealth with the issue is to totally block ZoneMinder logs, or to custom script to remove the mountainous error messages. Dennis Matveyev suggested a different method, where by modifying libjpeg to stop creating the warning messages in the source code and recompiling, effectively stopping the warning from ever being produced. His information is linked below, but is not ZoneMinder specific.

http://www-personal.umd.umich.edu/~dennismv/corruptjpeg.html

I thought I would give this method a go, as I did not simply want to block ZoneMinder from generating log messages, as this would be detrimental for debugging at later times. Stopping this warning message should be fine, as this computer is only used for ZoneMinder and as it states, its only a warning, it should not stop any functions. I first removed the associated libjpeg installations through apt-get, as a warning this will also remove ZoneMinder as it is dependent. This will delete all your monitors and settings! A work around would be to just force the removal of the one package. The following command should remove the package although depending on the revision it may have a different name:

sudo apt-get remove libjpeg8

It is then time to download the source for libjpeg, which can be found below:

http://www.linuxfromscratch.org/blfs/view/svn/general/libjpeg.html

Extract the files after the download. Before configuring or making the installation, the line which is creating the errornous warning needs to be commented out (or removed if you are a person who likes to live of the edge). The file for modification is jdmarker.c which needs the following line commented out:

WARNMS2(cinfo, JWRN_EXTRANEOUS_DATA, cinfo->marker->discarded_bytes, c);

jdmarker.c

Once that line is commented out, the usual ./configure / make / make install can be processed to install the library again. The next obvious step would be to install ZoneMinder again, unfortunately I found when the command apt-get install Zoneminder is processed, it wants to install libjpeg again, even though we just installed it, I still don't know many things about linux so I am not sure why. Luckily there is an easy way around this, forcing apt-get to download Zoneminder and then force the installation without worrying about the dependency with the following two commands:

apt-get download zoneminder
dpkg --force-all -i /var/cache/apt/archives/zoneminder_1.25.0-1_i386.deb  (Depending on where apt-get downloads the file).

If all has gone to plan, you should now have a warning free ZoneMinder log when using Foscam cameras. It seems to have been working hassel free for a while now for myself.

Comments (8) Trackbacks (1)
  1. Alternatively with syslog-ng, prevent logging those lines to file:

    destination df_null { };
    filter f_libjpeg { message(“WAR \\[Corrupt JPEG data: .+ extraneous bytes before marker”); };
    log {
    source(s_src);
    filter(f_libjpeg);
    destination(df_null);
    flags(final);
    };

  2. On CentOS I had to download the libjpeg tar, it gave me issues with it trying to call “./libtool” instead of the binary in my path (/usr/bin/libtool) so I just made a script that pointed everything to libtool
    echo “/usr/bin/libtool $@” >> /root/ZoneMinder-1.25.0/libtool; chmod +x /root/ZoneMinder-1.25.0/libtool

    Compiling libjpeg put the library files in /usr/local/lib/ instead of /usr/lib64. I backed up and moved the current libjpeg libraries (since doing a yum remove would remove a lot of needed dependencies).
    cd /var/lib64/
    tar cvzf /root/libjpeg.backup.tar.gz libjpeg*
    mv libjpeg* /tmp/
    ln -s /usr/local/lib/libjpeg* .
    Recompiled ZoneMinder and voila!

    Thanks for the write up!

  3. dvr@dvr:~/jpeg-8d$ sudo apt-get download zoneminder
    [sudo] password for dvr:
    Get:1 Downloading zoneminder 1.25.0-1 [1,849 kB]
    Fetched 1,849 kB in 0s (2,410 kB/s)
    dvr@dvr:~/jpeg-8d$ dpkg –force-all -i /var/cache/apt/archives/zoneminder_1.25.0-1_i386.deb
    dpkg: error: operation requires read/write access to dpkg status area
    dvr@dvr:~/jpeg-8d$ sudo dpkg –force-all -i /var/cache/apt/archives/zoneminder_1.25.0-1_i386.deb
    Selecting previously unselected package zoneminder.
    (Reading database … 35819 files and directories currently installed.)
    Unpacking zoneminder (from …/zoneminder_1.25.0-1_i386.deb) …
    dpkg: zoneminder: dependency problems, but configuring anyway as you requested:
    zoneminder depends on libjpeg8 (>= 8c); however:
    Package libjpeg8 is not installed.
    Setting up zoneminder (1.25.0-1) …

    Configuration file `/etc/init.d/zoneminder’
    ==> Modified (by you or by a script) since installation.
    Version in package is the same as at last installation.
    ==> Keeping old config file as default.
    Starting ZoneMinder: success

    Processing triggers for man-db …
    Processing triggers for ureadahead …

    So I followed the instructions but when I load zoneminder it still has my config and still has the error. Any ideas?

  4. Thanks for the steps, it helped updating the source package. For ubuntu I followed these steps as outlined on (http://www.moosechips.com/2008/09/ubuntu-rebuild-a-source-package/):

    cd /usr/src
    mkdir modjpeg;
    cd modjpeg
    sudo bash

    apt-get install build-essential fakeroot dpkg-dev devscripts

    apt-get source libjpeg-turbo
    cd libjpeg-turbo-1.1.90+svn733/
    vi jdmarker.c
    # Comment out the warning as outlined above #
    cd ..
    apt-get build-dep libjpeg-turbo
    cd libjpeg-turbo-1.1.90+svn733/
    dch -i
    # added my revision number and changes #

    dpkg-buildpackage -rfakeroot -uc -b
    cd ..

    # Install the new libjpeg and dependencies
    dpkg -i ./*.deb
    tail -f /var/log/syslog &
    service zoneminder restart

    # no more warning seen in syslog. :)

  5. Minimum number of commands:

    sudo apt-get install dpkg-dev
    sudo apt-get build-dep libjpeg-turbo
    apt-get source libjpeg-turbo
    cd libjpeg-turbo-1.1.90+svn733
    cp -p jdmarker.c jdmarker.c.orig
    vi jdmarker.c
    dpkg-buildpackage -rfakeroot -uc -b
    sudo dpkg -i ~zmadmin/libjpeg-turbo8_1.1.90+svn733-0ubuntu4.1_i386.deb
    sudo service zonemindesudo restart

  6. Dean’s example filter doesn’t work with the current syslog-ng versions, but I thought I’d pass along one that works ok for me.

    filter f_libjpeg { program(zmc_m2) and match(“extraneous bytes before marker” value(“MESSAGE”) ); };


Leave a comment