Back to the main page.
Bug 3044 - problem reading past the end of the file with EDF
| Status | CLOSED INVALID |
| Reported | 2016-01-15 09:30:00 +0100 |
| Modified | 2016-06-14 16:18:24 +0200 |
| Product: | FieldTrip |
| Component: | fileio |
| Version: | unspecified |
| Hardware: | PC |
| Operating System: | Mac OS |
| Importance: | P5 normal |
| Assigned to: | Robert Oostenveld |
| URL: | |
| Tags: | |
| Depends on: | |
| Blocks: | |
| See also: |
Robert Oostenveld - 2016-01-15 09:30:27 +0100
Arjen reported a problem reading all the way to the end of the file, which is fixed if he trims hdr.nSamples a bit. I received a 1GB large test file and will try to reproduce.
Robert Oostenveld - 2016-01-15 09:36:01 +0100
Yup, I also encounter the problem. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% >> hdr = ft_read_header('CA0081A7_1-1+.edf') Warning: Skipping "EDF Annotations" as continuous data channel because of inconsistent sampling frequency (270 Hz) > In read_edf (line 295) In ft_read_header (line 681) hdr = Fs: 1000 nChans: 81 label: {81x1 cell} nSamples: 7200000 nSamplesPre: 0 nTrials: 1 orig: [1x1 struct] chantype: {81x1 cell} chanunit: {81x1 cell} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% >> dat = ft_read_data('CA0081A7_1-1+.edf', 'chanindx', 1); Error using read_edf>readLowLevel (line 470) failed reading CA0081A7_1-1+.edf Error in read_edf (line 411) buf = readLowLevel(filename, offset, blocksize); % see below in subfunction Error in ft_read_data (line 622) dat = read_edf(filename, hdr, begsample, endsample, chanindx); %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% >> dat = ft_read_data('CA0081A7_1-1+.edf', 'chanindx', 1, 'begsample', 7200000-10, 'endsample', 7200000); Error using read_edf>readLowLevel (line 465) failed seeking CA0081A7_1-1+.edf Error in read_edf (line 411) buf = readLowLevel(filename, offset, blocksize); % see below in subfunction Error in ft_read_data (line 622) dat = read_edf(filename, hdr, begsample, endsample, chanindx);
Robert Oostenveld - 2016-01-15 10:35:19 +0100
the length of the file is inconsistent with the number of data blocks that it contains file size = 1117782016 bytes header size = 21248 bytes (according to the file itself) that leaves 1117760768 bytes for data. See http://www.edfplus.info/specs/edf.html The header specifies that there are 72000 blocks of 0.100 seconds each, which means that K>> floor(1117760768/72000) ans = 15524 K>> 1117760768 - 15524*72000 ans = 32768 there would be 32768 bytes not accounted for (assuming a block size of 15524). In fact the block size is sum(0.1*EDF.SampleRate) times two bytes (which means 100 samples per block for most channels and 27 samples for the annotation), which is 16254 samples. That means the file should be int32(16254*72000 + EDF.HeadLen) = 1170309248 bytes long. So there are 1170309248 - 1117782016 = 52527232 bytes missing. Let me parse the actual and expected file size with thousand separators: 1.170.309.248 expected 1.117.782.016 actual 52.527.232 difference so there are about 52 MB missing at the end of the file. Either the header was incorrectly written, or the file was truncated somehow. In short: the file is corrupt. In principle FT could be made to deal with that by adjusting the EDF.NRec field (and giving a warning). Since you have a solution and since it seems a one time corruption, I will not bother with that myself.
Arjen Stolk - 2016-01-15 19:30:03 +0100
Thanks so much for looking into this! It remains strange that the same file could be read a couple of months ago. Assuming nothing changed in the code, it may well have been, as you suggest, that the file became corrupted by copying over from one place to another. Let's assume this for the moment. Thank you very much again.
Arjen Stolk - 2016-01-15 19:59:08 +0100
Just managed to get access to the original file which was indeed slightly smaller than the one we've been looking at. So you were right - as usual - in that this may have been caused by a copy-error.
Robert Oostenveld - 2016-01-17 10:23:15 +0100
(In reply to Arjen Stolk from comment #4) > So you were right - as usual - :-P