Is there a log file joining utility...?

[swingtan]

Hmmmm I didn't actually check that. I'm looking at it now and is seems to be running at normal speed for me.

I'm now looking at compiling the PERL code so it'll run without the need to install PERL ( though the *nix uses will already have that ).

Simon.

[vxchev8]

Nice work Simon.

[swingtan]

OK, version 0.1 is available for Beta testing......

Remember: This is an "all care, no responsibility" production....

I've tested it on both E38 and LS1 log files and it works OK. It should be fine for the other controllers as long as...

• The 1KB header is std.
• The metadata section starts and ends the same way.
• There are no additional data sections in the log file ( apart from the actual log data).

Basics:

This utility ONLY works from the command line. If you are not comfortable using the command prompt / DOS window / ETC. then this is not for you. While the utility is only meant to work on EFILive log files, it will happily write to any file named. It is possible to overwrite files that you didn't actually want over written. PEBCAK is the responsibility of the user....

Set up:

simply unzip the file and save it to where ever you want. You can save it to C:\Windows or C:\Windows\system32 and it will be in your default path for the command prompt. If you put it somewhere else, you may need to add that directory to the path variable or use the full path when running the tool. I'd stick it in C:\Windows, unless there is already a file there called "join.exe"

Usage:

Start a command prompt window.

click on [start] - [run]
type in "cmd" ( no quotes )
then click on [OK]

Then change directory to where you have stored your log files. The STD. directory is under your "My Documents" folder.

cd "My Documents"\EFILive\V7.5\"Logged Data"

Next identify the 3 required parameters for the tool...

1. Source File: This file is used for the PID description and it's data will appear first in the final log.
2. Additional Files: the names of each file that you want to add to the source file. They are added in series so you can set the order you want.
3. Output File: This is what the final joined log file will be called.
   WARNING: This tool will overwrite any file named as the output file if it exists. It will not ask for confirmation.

You can run the tool in it's basic form....

join -s [source file] -a [add file-1] ... -a [add file-n] -o [output file]

It will take a couple of seconds to set up the environment, and then output some details on the screen as to what is going on. A sample screen will look like this...

 > join -s BBL00119.efi -a BBL00120.efi -a BBL00121.efi -o test.efi

Source file: BBL00119.efi
Source size: 51637
Source Meta size: 613

        Next File: BBL00120.efi
                Next size: 51637
                Next meta: 613
        Next File: BBL00121.efi
                Next size: 52137
                Next meta: 613

Completed join to output file test.efi

There are 3 other command line switches.....

1. -h : The "help" option, which gives some tips on the tool and how to use it.
2. -v : Verbose mode, which just prints the metadata data as found in the source file.
3. -d : Debug mode, which does everything EXCEPT write to the output file. This mode is useful when combined with "-v" to list the metadata if you want to check things like the PID selections.

So, if you feel up to it, try it out and see if it is of any use.

Simon.

[Blacky]

My update display frequency is 2, but this log plays faster like it is set at 1.
I set it to 4 and it played back at the speed of 2.
Is this a side effect or do logs have the display frequency stored in them?

The playback speed is based on the timestamps in the frames in the log file. It is anchored off of the first frame. I guess if the timestamp of the frames in the appended file are later then the actual file, then EFILive will insert a configurable delay (Properties->Playback->Frame times->Max pause) whenever it encounters two frames with a gap larger than the configured time delay between them.

If two frames have a gap that is negative, i.e. the second frame has an earlier timestamp then the first frame then I don't know what will happen. I never coded for that possibility. (I figured backward time travel had not been invented yet so I would be safe with that assumption).

I think what would happen it the gap between the two frames would blow out to a few years (around 2^31 seconds or about 68 years) and EFILive would just make the delay between the frames whatever Max pause is set to.

Regards
Paul

[mr.prick]

Like I said my settings for playback are a little slower than real time,
I opened this file and it played back in real time regardless of my settings.
I found this odd.

Looking at the log again I see that 30 channels are selected too,
making this all the more spooky.

[swingtan]

E38....... It's amazing how much data you can log

[vxchev8]

Nice work Simon - It allowed me to join two logs without any problems.

I put the join.exe file into the windows folder then simply followed your instructions.

[swingtan]

Hi All,
I'm part way through the next version that will hopefully work on specific PID's. It should be able to pull a set list of PID's from multiple files and join them together.

Here's a quick screen dump....

swingtan > logtool.pl -s small.efi -f

Source file: small.efi
Source size: 4031
Source Meta size: 1957

Channel Count: 42
Frame size: 50
Data length: 1050
Frame Data:

49 a3 2b 3b 00 3c 03 00 00 01 8e 85 2f 85 00 00 00 03 82 0c 86 05 68 80 80 0c 39 65 d4 00 00 01 03 7d 05 20 00 00 06 a5 09 c4 00 00 00 00 c7 9a 93 03
49 a6 2b 3b 00 3c 06 00 00 01 8e 85 2f 85 00 00 00 03 8a 0c 92 05 6b 80 80 0c 3e 65 d4 01 00 01 03 7d 05 20 00 00 06 a5 09 c4 00 00 00 00 e6 9a 93 03
49 a6 2b 3b 00 3c 06 00 00 01 9b 85 2f 8c 00 00 00 03 86 0c 97 05 6a 80 80 0c 68 65 be 03 00 01 03 7d 05 20 00 00 06 a5 09 c4 00 00 00 00 05 9b 93 03
49 a2 2b 3b 00 3c 09 00 00 01 95 85 2f 8c 00 00 00 03 93 0c 9f 05 6a 80 80 0c 73 66 00 05 00 01 03 7d 05 20 00 00 06 a5 09 c4 00 00 00 00 25 9b 93 03
49 a1 2b 3b 00 3c 07 00 00 01 95 85 2f 8c 00 00 00 03 93 0c 9f 05 6d 80 80 0c 85 65 d4 06 00 01 03 7d 05 20 00 00 06 a5 09 c4 00 00 00 00 44 9b 93 03
49 a1 2b 3b 00 3c 06 00 00 01 8e 85 2f 8c 00 00 00 03 8d 0c 8c 05 6e 80 80 0c 96 66 00 00 00 01 03 7d 05 20 00 00 06 a5 09 c4 00 00 00 00 63 9b 93 03
49 9d 2b 3c 00 3c 06 00 00 01 8e 85 2f 8c 00 00 00 03 8a 0c 92 05 70 80 80 0c 85 65 d4 02 00 01 03 7d 05 20 00 00 06 a5 09 c4 00 00 00 00 82 9b 93 03
49 9f 2b 3c 00 3c 06 00 00 01 88 85 2f 8c 00 00 00 03 7f 0c 82 05 71 80 80 0c 84 65 be 03 00 01 03 7d 05 20 00 00 06 a5 09 c4 00 00 00 00 a1 9b 93 03
49 9f 2b 3c 00 3c 04 00 00 01 88 85 2f 8c 00 00 00 03 7a 0c 80 05 6e 80 80 0c a0 66 16 05 00 01 03 7e 05 22 00 00 06 a5 09 c4 00 00 00 00 c0 9b 93 03
49 9a 2a 3c 00 3c 04 00 00 01 88 85 2f 8c 00 00 00 03 81 0c 85 05 6a 80 80 0c b3 66 00 07 00 01 03 7e 05 22 00 00 06 a5 09 c4 00 00 00 00 df 9b 93 03
49 99 2a 3d 00 3c 04 00 00 01 88 85 2f 92 00 00 00 03 7e 0c 80 05 66 80 80 0c ba 66 2c 00 00 01 03 7e 05 22 00 00 06 a5 09 c4 00 00 00 00 fe 9b 93 03
49 98 2b 3d 00 3c 04 00 00 01 88 85 2f 92 00 00 00 03 7e 0c 77 05 65 80 80 0c a2 66 16 02 00 01 03 7f 05 23 00 00 06 a5 09 c4 01 00 00 00 1d 9c 93 03
49 9b 2b 3d 00 3c 03 00 00 01 8e 85 2f 92 00 00 00 03 79 0c 79 05 65 80 80 0c 7c 66 00 04 00 01 03 80 05 25 00 00 06 a5 09 c4 00 00 00 00 3c 9c 93 03
49 a0 2b 3b 00 3c 03 00 00 01 88 85 2f 92 00 00 00 03 81 0c 85 05 65 80 80 0c 78 66 16 05 00 01 03 81 05 26 00 00 06 a5 09 c4 00 00 00 00 5c 9c 93 03
49 a0 2b 3b 00 3c 03 00 00 01 88 85 2f 92 00 00 00 03 7a 0c 77 05 63 80 80 0c 63 66 00 07 00 01 03 81 05 26 00 00 06 a5 09 c4 00 00 00 00 7b 9c 93 03
49 a3 2a 3b 00 3c 03 00 00 01 88 85 2f 92 00 00 00 03 7f 0c 81 05 65 80 80 0c 56 66 2c 01 00 01 03 81 05 26 00 00 06 a5 09 c4 00 00 00 00 9a 9c 93 03
49 a3 2a 3b 00 3c 03 00 00 01 8e 85 2f 92 00 00 00 03 81 0c 85 05 65 80 80 0c 8d 66 00 02 00 01 03 82 05 28 00 00 06 a5 09 c4 00 00 00 00 b9 9c 93 03
49 a2 2b 3b 00 3c 03 00 00 01 88 85 2f 92 00 00 00 03 80 0c 7d 05 61 80 80 0c 9d 66 43 04 00 01 03 84 05 2b 00 00 06 a5 09 c4 00 00 00 00 d8 9c 93 03
49 9c 2a 3b 00 3c 06 00 00 01 88 85 2f 98 00 00 00 03 7c 0c 7d 05 62 80 80 0c ae 66 2c 06 00 01 03 84 05 2b 00 00 06 a5 09 c4 00 00 00 00 f7 9c 93 03
49 9a 2b 3d 00 3c 04 00 00 01 88 85 2f 98 00 00 00 03 7c 0c 7d 05 5e 80 80 0c 95 66 16 07 00 01 03 86 05 2d 00 00 06 a5 09 c4 00 00 00 00 16 9d 93 03
49 9d 2b 3d 00 3c 04 00 00 01 88 85 2f 98 00 00 00 03 7b 0c 80 05 61 80 80 0c 87 66 2c 01 00 01 03 86 05 2d 00 00 06 a5 09 c4 00 00 00 00 36 9d 93 03

[joecar]

Simon, good job mate...

[vxchev8]

...