Printable Version of Topic

Posted by: Lawrence Aug 9 2004, 09:18 AM

Sorry for the OT post.

I have a text file that I need to edit.

I need to edit every record on that file (record positions 34-37) to change whatever is in those positions (only) to a four digit number. What is currently in those positions varies dramatically.

My only text editor I have on the system is vi.

Anyone got a command I can use?

thanks,
Lawrence

Posted by: fiid Aug 9 2004, 09:46 AM

hey,

It sounds like you need to make this edit on a lot of lines?

You might be better off using a small SED or AWK (or perl if you have it) script to do the job.

Another option is simply download the file, and use something on a box with more tools on it. Post some more details and I can help you get a script together.,

Fiid.

Posted by: thomasotten Aug 9 2004, 10:04 AM

upload the text into a database like mysql, if you have it. Then use sql to change it. it would be quicker than trying to figure out awk and sed. I hate those tools.

Posted by: sgomes Aug 9 2004, 10:05 AM

Wow! Two things:

1) vi is still getting discussion!

2) It's getting discussion on a 914 website!

Posted by: mikester Aug 9 2004, 10:07 AM

If the records were all the same it would be easy in vi but since they aren't...that's another story.

Can you give me a sample line from the file? Fake data is fine - just show me the format.

Posted by: vortrex Aug 9 2004, 10:09 AM

are you using real vi, as in on a unix station of some sorts?

Posted by: Lawrence Aug 9 2004, 10:17 AM

Here's a sample record:

000104659 W91XLM40940026 GSA00001000011BA042200418504208W91XLM40940026XXXBY

I need to replace the four characters after BA0... "4220".

It shouldn't matter, but that's a julian date.

Unfortunately, the only option I have is to use the existing system (with real VI, on a stripped down version of Solaris), unless I want to use get/dosput to bring them over to my Windows system. I dunno copying them back will screw up their formatting.

thanks for the input (no pun intended!)

Posted by: Root_Werks Aug 9 2004, 10:39 AM

Oh man, it has been a while. But even Solaris should have sed:

sed /BOA2440/BAO9999/g

Is that it? Man, I need to fire up my linux box and play around some again. First string you want to look for, second one whatever you want to change it to and the "g" for global.



I think?

Posted by: Lawrence Aug 9 2004, 10:48 AM

The BA0 only varies a bit. I could run several swaps if it was that simple.

However, the 4220 is a date, and I have a range in that field from 3351 to 4222.

P.S. I'm working with ~2000 records. I'm not good enough with vi (or fast enough) to edit each record individually. I'm trying to make this little program do something it's not designed to do.

Posted by: thomasotten Aug 9 2004, 10:54 AM

Ok, how about writting a shell script that reads every line, then prints out the concatination of the prefix string, with your replacement string, then the remainder string

I just tried it:

test.sh
----------------------------------

s="000104659 W91XLM40940026 GSA00001000011BA042200418504208W91XLM40940026XXXBY"
echo ${s:0:42}"REPL"${s:46}

------------------------------------------
run it by typing "sh test.sh"

"REPL" above is the replacement text

Now you just need to make the script read the file, and do this same operation on every line.

Posted by: kafermeister Aug 9 2004, 11:00 AM

Ok, Don't laugh. The last time I ran into something like this, I was using SecureCRT to console into a Linux box. I copied the contents of the file (although not 2000 lines) from SecureCRT into Notepad and did a replace all. I backed up my original file on the Linux machine and opened it. Deleted all lines (holding 'd' for many minutes), went into insert mode and pasted the whole shebang from Notepad once it was modified. I didn't want to re-create the file on the Linux box because of possible ownership and permissions issues.

Sorry, I love Unix like OS but I put food on the table supporting Windows.

HTH
Rick

Posted by: Lawrence Aug 9 2004, 11:01 AM

Thomas,

The rest of the data is different in every record. Will the script wildcard the rest of the record?

-Lawrence

Posted by: thomasotten Aug 9 2004, 11:04 AM

yeah, it is just looking for the same position. It is counting chars up to that point, poping in your replacement, then tagging on the remainder string. Make yourself that test.sh file like I showed, and try to run it to see.

Posted by: mikester Aug 9 2004, 11:18 AM

The simplest way to do this is in vi.

If it's only dates between 3351 to 4222 then maybe you could try the following.

in vi type the following;

:%s/3351/newstring/g

Then press enter, that will replace all of the 3351's in the file with the new string - if 3351 occurs anywhere else in the line though it will be replaced as well.

You could do this for each date you want to change - depending on the number of entries it's relatively quick.

YMMV

Posted by: tat2dphreak Aug 9 2004, 11:56 AM

ok, if this is on a windows machine that you are telenting or connecting to UNIX through:
threre is a great program called Ultraedit... you can edit all the lines at once...

if not you can do something like this as well

for line in "do
sect_1=$(echo $line |cut -c 1-41)
old_data=$(echo $line |cut -c 42-46)
sect_3=$(echo $line |cut -c 47-)
new_data='4225'


print "${sect_1}${new_data}${sect_3}\n"
done >> new_file_name

Posted by: fiid Aug 9 2004, 12:16 PM

Oh man, it has been a while. But even Solaris should have sed:

sed /BOA2440/BAO9999/g

Is that it? Man, I need to fire up my linux box and play around some again. First string you want to look for, second one whatever you want to change it to and the "g" for global.



I think?

Posted by: opera guy Aug 9 2004, 12:47 PM

holy crap, all the unix hackers are coming out of the woodworks in the 914 forum.

Posted by: Root_Werks Aug 9 2004, 02:23 PM

QUOTE(Root_Werks @ Aug 9 2004, 08:39 AM)

  Oh man, it has been a while.  But even Solaris should have sed:

sed /BOA2440/BAO9999/g

Is that it?  Man, I need to fire up my linux box and play around some again.  First string you want to look for, second one whatever you want to change it to and the "g" for global.



I think?

you can change this to match any 4 numbers by doing something like:

/BAO[0-9]{4,4}/BAOxxxx/g

(You can probably refine this a bit - read the man page or appropriate o'reilly book)

Fiid.

Posted by: fiid Aug 9 2004, 04:24 PM

We should start a 914 Wiki is what we should do.

Posted by: Gint Aug 9 2004, 05:15 PM

Get this figured out yet? Here's a very simple solution and explanation. This assumes that each line only contains one "BA0...." string. Otherwise we'll have to use some of the positional commands some of these other "hacks" (and I mean that in the nicest way possible ) listed. Also, all of this changes if the old replacement string has a data value that needs to relate to the replacement string. If the replacement string for every line will be the same, then it's this simple. If not, we have to be more creative.

cat tmpfile | sed 's/BA0..../BAO1234/g' > tmpfile_new

cat your file (tmpfile in this example), pipe it to sed. the "s" means substitute. Your sub-ing BA.... (that's four dots, or wildcards for one character each in regular expression speak) for the char set infollowing the next forward slash. In this case you would want to replace them with the sam BA0 and then the 4 character string you want to use as a replacement string. I used 1234 in this example. The > re-directs the new text to a new file, leaving the existing file intact. You can then rename the new file as necessary.

If you just run this

cat tmpfile | sed 's/BA0..../BAO1234/g'

without the file re-direct, you can see the output on your terminal display and verify it's what you want before creating the new file.

This should work on any Solaris system (I tested it on one to make certain). The systems standard path should have the location of sed and cat.

Posted by: Lawrence Aug 9 2004, 09:44 PM

Thanks guys! It's morning here now... I'm going to give it another shot!

-Rusty

Posted by: Gint Aug 9 2004, 10:05 PM

Here's an even shorter version:

sed -e 's/BA0..../BAO1234/g' > newfilename


BTW - There is almost always at least 14 different ways to do the same thing in Unix. Pick one you like the best and go with it.

Another thing - I finally met Brad Mayeur Friday night. Told him you said "Hi."

Rack time...

Posted by: Root_Werks Aug 10 2004, 11:53 AM

Me thinks Gint wins the prize for this one!

Posted by: bperry Aug 10 2004, 12:35 PM

Man its been a while but I love vi and regular expressions.
We used to have contests like this at work back in the 80's
to see who could do it in the shortest (most obtuse) way.

Its been years since I've done any of this
but you could try something like this from the command line.

:1,$s/^\(..... place 32 dots here\)....\(.*\)/\1####\2/

This should modify the entire file.

There should be EXACTLY as many periods/dots between the
first set of parens as you want to skip over.
Then EXACTLY as many periods/dots between the first
set of parens and second parens as you want to replace.
Replace #### with the new numbers/text you want.
(doesnt have to be the same length as what is being replaced)
This works by matching the begining characters and puting them in a substring
then matching the replacement characters and finally puting the tail
end of the string in another stubstring. Then it puts the first
substring back, then the new text, followed by the second substring.

Hopefully, I remembered correctly and the WEB tools didn't mangle
any of the special characters.


(NOTE: I did download a vi versions for windows and this does work.)
Regular expressions are cool! and very consistent across unix.

For all of you wondering how come I got so deep into such
techno dweeb stuff, its because back in the "old days" when you
were on a 300 baud modem, you wanted as little screen activity
as possible and the ":" commands of vi are exactly the same as
the "ed" commands which is where vi came from.

--- bill