fprot.sh bug: $URL_UPDATE_SITE has multiple lines

Report new bugs here, or look at known issues of current and previous releases of UBCD.

Moderators: Icecube, StopSpazzing

Post Reply
Message
Author
Explorer09
Posts: 178
Joined: Fri Jun 17, 2011 11:23 pm

fprot.sh bug: $URL_UPDATE_SITE has multiple lines

#1 Post by Explorer09 » Fri Aug 10, 2012 11:01 pm

I found recently that the fprot-update-link.xml file from f-prot website is formatted in one line. And this makes the script ubcd/tools/linux/antivirus/fprot.sh not working.

How to test:
1. Run the ubcd/tools/linux/antivirus/fprot.sh script. (in UBCD 5.1.1)

Actual result:
The script will fail to download the antivir.def file, and you'll see the error message like below.

Code: Select all

Start downloading the F-Prot definition file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

--2012-08-11 13:54:35--  http://srv3.directupdates.f-prot.net%0Asrv6.directupdates.f-prot.net%0Asrv7.directupdates.f-prot.net%0Asrv8.directupdates.f-prot.net%0Asrv9.directupdates.f-prot.net/files/defs/v.6/2012-08-10T14-45-01-673e9e0b9fd99bbff4dfc64b87c5c8f4/antivir.def
Resolving srv3.directupdates.f-prot.net\nsrv6.directupdates.f-prot.net\nsrv7.directupdates.f-prot.net\nsrv8.directupdates.f-prot.net\nsrv9.directupdates.f-prot.net (srv3.directupdates.f-prot.net\nsrv6.directupdates.f-prot.net\nsrv7.directupdates.f-prot.net\nsrv8.directupdates.f-prot.net\nsrv9.directupdates.f-prot.net)... failed: Name or service not known.
wget: unable to resolve host address `srv3.directupdates.f-prot.net\nsrv6.directupdates.f-prot.net\nsrv7.directupdates.f-prot.net\nsrv8.directupdates.f-prot.net\nsrv9.directupdates.f-prot.net'

Download finished.


Make 'fprot-definitions.txz' file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
tar: /tmp/ubcd-temp/f-prot/definitions: Cannot chdir: No such file or directory
tar: Error is not recoverable: exiting now
(Note the line:
wget: unable to resolve host address `srv3.directupdates.f-prot.net\nsrv6.directupdates.f-prot.net\nsrv7.directupdates.f-prot.net\nsrv8.directupdates.f-prot.net\nsrv9.directupdates.f-prot.net')

Expected result:
The script should successfully locate the antivir.def file and download it.

I made a patch for this script. I added a

Code: Select all

head -n 1
command after grep so that the variable $URL_UPDATE_SITE will always contain one line.
http://dl.dropbox.com/u/70170658/fprot-patched.sh

EDIT: Now I use

Code: Select all

| sed -n '1 p'
instead, because UBCD does not have the win32 binaries for "head" command.
Last edited by Explorer09 on Tue Sep 11, 2012 6:13 pm, edited 2 times in total.

ady
Posts: 832
Joined: Sat May 08, 2010 5:26 am

Re: fprot.sh bug: $URL_UPDATE_SITE has multiple lines

#2 Post by ady » Sat Aug 11, 2012 12:41 am

Whether under Linux or Windows, it's not working for me :(. Sorry I can't give you suggestions about it.

(BTW, other scripts from UBCD v5.1.1 need some adjustments / updates too, but that's for another topic.)

Explorer09
Posts: 178
Joined: Fri Jun 17, 2011 11:23 pm

Re: fprot.sh bug: $URL_UPDATE_SITE has multiple lines

#3 Post by Explorer09 » Sat Aug 11, 2012 4:07 am

Sorry. I forget to tell that you should put the script into the original directory. I mean here: <ubcd-extracted>/ubcd/tools/linux/antivirus/

Otherwise the script won't run, just like the original one.

ady
Posts: 832
Joined: Sat May 08, 2010 5:26 am

Re: fprot.sh bug: $URL_UPDATE_SITE has multiple lines

#4 Post by ady » Sat Aug 11, 2012 6:52 am

Yes, of course, but it still fails. I know I should present more details, but I don't really have any right now. I mean that the script runs, but the definitions are not really downloaded, and I end up with a 1KiB txz, at best. This happens under both Linux and Windows. I guess I would need to test what's different in my systems and my customized UBCD.

ady
Posts: 832
Joined: Sat May 08, 2010 5:26 am

Re: fprot.sh bug: $URL_UPDATE_SITE has multiple lines

#5 Post by ady » Sat Aug 11, 2012 9:24 am

Your second revision using sed finally worked (thank you for sharing), but I should add some comments.

The general steps are:

1._ In fprot.sh, replace the original line from UBCD511:

Code: Select all

URL_UPDATE_SITE=`grep -m 1 -o srv[[:digit:]].directupdates.f-prot.net "${FPROT_TEMPDIR}/fprot-update-link.xml"`
with the suggested (also in one line):

Code: Select all

URL_UPDATE_SITE=`grep -m 1 -o srv[[:digit:]].directupdates.f-prot.net "${FPROT_TEMPDIR}/fprot-update-link.xml" | sed -n '1 p'`
and save it. The name and directory of the script should remain exactly the same as before.

2._ Search for the "ubcd-temp" directory and clean up any residual files, including files in any subdirectories. I clean up the files only, and I leave untouched the (sub)directories themselves. Cleaning up the files only and leaving the directory tree helps reduce some potential errors I found during my tests.

3.A._ If using Windows, go to the folder where fprot.cmd is located and open a cmd prompt window in there.
3.B._ If using Linux, go to the directory where fprot.sh is located and open a terminal window in there.

4.A._ If using Windows, type in

Code: Select all

fprot
and press <enter>.
4.B._ If using Linux, the instructions may vary depending on your specific distro. In my case, I type in

Code: Select all

su <enter>
password: <type in your password>
bash ./fprot.sh
5._ Once the definitions are downloaded and updated, type in

Code: Select all

exit <enter>
. In some cases, this step may be needed more than once.


Some additional comments...

I don't know why I needed to run the update as superuser under Linux, but while using my normal user the script failed one way or another. Other distros may not require this condition, and others may use "sudo" instead. YMMV. Ideally, the scripts should also be tested under PartedMagic itself from a UFD.

I will also reproduce here what's already written in the relevant readme.txt for the Linux-based scripts:
If you have extracted the iso and if you want to run the scripts in the subfolders,
make sure that they are executable. This is only needed if you run GNU/Linux.

Change to the directory (cd) of this readme file and run:

Code: Select all

  chmod a+x ./*/*.sh
  chmod a+x ./ubcd2usb/syslinux
  chmod a+x ../../../antivir/antivir
I'd like to point out that this script only updates the definitions for fprot; not the program itself. Additionally, I think there is some space for more improvements (and corrections) to the scripts in general. Hopefully they will be applied in future releases of UBCD.

Explorer09
Posts: 178
Joined: Fri Jun 17, 2011 11:23 pm

Re: fprot.sh bug: $URL_UPDATE_SITE has multiple lines

#6 Post by Explorer09 » Wed Aug 22, 2012 4:49 am

Third revision!

Since F-Prot has multiple servers for me to download the virus definitions file, I took its advantage and added a loop to try every server to download the file.

The patched script can be downloaded here (same location):
http://dl.dropbox.com/u/70170658/fprot-patched.sh

Because I changed many lines of code for this revision, I will just show the diff below without explanation.

Code: Select all

--- fprot.sh	2011-04-10 23:05:34.000000000 +0800
+++ fprot-patched.sh	2012-08-22 20:30:39.877270876 +0800
@@ -9,11 +9,15 @@
 
 update()
 {
-    rm -f "${FPROT_TEMPDIR}/definitions/opt/f-prot/antivir.def"
+    # Save IFS
+    SAVED_IFS=$IFS
+    # Make IFS a empty string. Otherwise it will convert newlines to spaces 
+    # in $URL_UPDATE_SITES variable
+    IFS=
 
     # Get the link to the definition file
     # Note: it is necessary to enclose '/' in square brackets for bash/Win32
-    URL_UPDATE_SITE=`grep -m 1 -o srv[[:digit:]].directupdates.f-prot.net "${FPROT_TEMPDIR}/fprot-update-link.xml"`
+    URL_UPDATE_SITES=`grep -o srv[[:digit:]].directupdates.f-prot.net "${FPROT_TEMPDIR}/fprot-update-link.xml"`
     URL_UPDATE_PATH=`grep -m 1 -o [/]files[/]defs[/].*[/]antivir.def "${FPROT_TEMPDIR}/fprot-update-link.xml"`
 
     echo "Start downloading the F-Prot definition file"
@@ -21,7 +25,25 @@
     echo
 
     # Download antivir.def file
-    wget -P "${FPROT_TEMPDIR}/definitions/opt/f-prot/" "${URL_UPDATE_SITE}${URL_UPDATE_PATH}"
+    while [ -n "${URL_UPDATE_SITES}" ]; do
+        URL_UPDATE_SITE=`echo "${URL_UPDATE_SITES}" | sed -n '1 p'`
+        URL_UPDATE_SITES=`echo "${URL_UPDATE_SITES}" | sed '1 d'`
+        rm -f "${FPROT_TEMPDIR}/definitions/opt/f-prot/antivir.def"
+        wget -P "${FPROT_TEMPDIR}/definitions/opt/f-prot/" "${URL_UPDATE_SITE}${URL_UPDATE_PATH}"
+        if [ "$?" -eq "0" -a -f "${FPROT_TEMPDIR}/definitions/opt/f-prot/antivir.def" ]; then
+            DOWNLOADED="true"
+            break
+        fi
+    done
+
+    if [ "$DOWNLOADED" != "true" ]; then
+        echo
+        echo "ERROR: Couldn't download 'antivir.def' file from any of the F-Prot servers."
+        exit 1
+    fi
+
+    # Restore IFS
+    IFS=$SAVED_IFS
 
     echo
     echo 'Download finished.'
Diff file can be downloaded here:
http://dl.dropbox.com/u/70170658/fprot-patch.diff

ady
Posts: 832
Joined: Sat May 08, 2010 5:26 am

Re: fprot.sh bug: $URL_UPDATE_SITE has multiple lines

#7 Post by ady » Wed Aug 22, 2012 7:39 am

For some reason, I thought that the download site was already "random", taken from the several servers available.

Anyway, I am unable to test this. "Something" happened in my Windows OS, and the scripts are not working, at all (not even the older ones, that used to work some days ago). The error "seems" to be related to cygwin, but the original scripts, using the original auxiliary files of UBCD511, are not working either (not even failing with the same type of errors as before the patches).

Now I wonder if some software in my system have recently changed something about cygwin, or maybe some recent Windows update (2012AUG) is generating these incompatibility problems.

Am I the only one with these issues? Can anyone please confirm? I'll provide more details if necessary.

Post Reply