[ILUG] Re: perl file processing

[Marcus Furlong]

Brian Foster <blf <at> utvinternet.ie> writes:

> 
>  below's a quickly-put-together all-awk(1) solution,
>  albeit if this was my problem I'd be more inclined
>  to do some filtering first, probably with sed(1)
>  like Francis did.
> cheers!
> 	-blf-
> 
> #!/bin/gawk -f
> BEGIN {
> 	state  = 0
> 	ncols  = 0
> 	nlines = 0
> 
> 	STDERR = "/dev/stderr"
> }
> 
> state == 0 && $0 == "=== Stratified cross-validation ===" {
> 	state = 1
> 	next
> }
> 
> state == 1 && $0 == "=== Detailed Accuracy By Class ===" {
> 	state = 2
> 	next
> }
> 
> state == 2 && 2 <= NF && $NF ~ /^[A-Z]$/ {
> 	for (n = 1; n < NF; n++) {
> 		if ($n !~ /^[0-9.]*$/)
> 			next
> 	}
> 	state = 3
> 	ncols = NF
> }
> 
> state == 3 && NF != ncols { exit }	# goto END
> 
> state == 3 && $NF !~ /^[A-Z]$/ { exit }	# goto END
> 
> state == 3 {
> 	for (n = 1; n < ncols; n++)
> 		col[n] += (0 + $n)
> 	nlines++
> 	next
> }
> 
> END {
> #debug	print "EXIT(" FNR "): nlines=" nlines, "ncols=" ncols
> 	if (nlines <= 0) {
> 		print FILENAME ": Data not found, state =", state  >STDERR
> 		exit 1
> 	}
> 	for (n = 1; n < ncols; n++)
> 		print col[n]/nlines
> }
> 

Thanks for this one too.

I had to change [A-Z] to [a-zA-Z]* as the class is not always a single uppercase
character. I ran both yours and Francis' from a shell script over the 250 files,
and out of curiosity, I checked the times for each.

The sed/awk one comes out with the following:
real    0m32.733s
user    0m10.915s
sys     0m11.838s

whereas the pure awk one is a lot faster:
real    0m18.239s
user    0m2.035s
sys     0m2.761s

Ran a few times just in case, and those times are fairly consistent. Anyway
thanks to you both!

Marcus.