Drivewire over Ethernet
The Drivewire protocol is a means of disk, time, printer, and tcp offloading designed for the Tandy Color Computer (CoCo) series by Boisy Pitre, and later extended a bit by Aaron Wolfe. Drivewire is usually implemented on top of a streaming UART interface (or the Color Computers Bitbanger) as a stream oriented protocol.
In a recent CoCoCrew Podcast (shameless brown-nosing the judge), John Linville noted that he found this project, and thought it would be neat to make something like this for the CoCo. I think so too, and I'm one of a very few CoCo retro-programmers who owns a prototype of Jim Brain's NIC card for the CoCo.
So, for RetroChallenge 4/2017, I will be combining the Drivewire protocol and my CoCoNIC card, and seeing what happens. Stay tunes!
In a previous posting, I established the roles of the Coco vs. The drivewire server. All communications are Client initiated, making the CoCo the 'master' device, and the Drivewire Server the 'slave'. Now lets look closer at the communications happening during each kind of transaction, or discrete unit of command/response:
Unidirectional: coco sends a command to the dw server. (Lpr, old init, vport output)
Bidirectional: coco sends a command and expects a result. (disk service, time, new init)
Multi-transactional: two bidirectional transfers (only disk read extended)
Bidirectional transactions fit well into a packets. The other two types, not so much.
Unidirectional transactions assume reliable communications from the coco to the server. This make good sense, as traditional dw servers connected via a UART, no doubt, have receive buffers. The same can't be said of ethernet packets. Ethernet packets are subject to switching, buffering, lags, dropping, mangling, duplication, line collisions, etc. If we dont expect a reply from the server, then we cant be really sure the server got our command packet. We'll have to change this behavior to have the server send a reply packet every transaction. Even if its data length is zero, a response packet is needed from the server to confirm the command reception.
There is only one multi-transactional command in the current Drivewire: The 'extended read command introduced to replace 'read'. This is a necessary improvement considering the original drivewire protocol that operated at 115.2k (or 56k on coco2), not allowing for much (if any) time for the coco to make decisions based on the server's response *and* receive bits. For packetization, this "double transaction" hurts us though, as it leaves the server in a different state between the two transactions. The server now expects the client to send it something before going back to its general, beginning state. A lost client packet may lock the server. Server timeouts could be done, but this adds complexity and would lock out *other* clients, possibly timing them out. My spider sense is tangling: "complexity bad!", it says. Nope: too far down the rabbit hole for me, so I will choose to prohibit a statey server by disallowing the READ_EXT operation. The original READ op will work fine though for packetization. Also, our hardware communication (a CS8900 NIC) has full buffering, allowing our client to take its time reading (and deciding on) the servers resonse.
Now that we have a rough version of a working packet driver for client and server. Next I'll start on adding an actual drivewire server code. For this I'll use William Astle's 'LW Wire" server. LWWire is written in plain-jane C, and is well commented. So patch primary is my work to switch C's "main()" over the lwwire, and redo packet.c's "main()", as an device initialization method, instead. I also roughed out a software interface to packet.c:
I'm not particularly sure this is the interface we'll end up with, but we'll fine tune later.
LW Wire's primary communications device is UNIX's stdin and stdout. This works fine for standard DriveWire as it expects a serial stream of bytes. DriveWire over Ethernet, however we'll want to rip out the calls to io read/write, and insert our packet interface from above.
To make server debugging easier, I also modified our CoCo Client code to repeatively:
send a drivewire "time" operation.
receive a valid reponse frame.
print valid reponse frame.
wait for keypress.
And the results on the CoCo are pleasing, as this represents a almost usefull packet exchange:
DATA 0-? bytes of actual DW command data
OK. Last night I conjigured (real word?) the CoCo "client" to receive Eth0 frame from the CoCoNIC Card. Again, this was just following along with Fuzix's Driver in C, so nothing earth shattering redoing it in 6809 Asm. The CoCoNIC card is setup to receive any frames matching our MAC address or and Broadcast frames (desc MAC = ff:ff:ff:ff:ff:ff). I spend a few minutes adding some filters on the CoCo end to filter out packets that we don't care about:
Get a Frame from NIC
Not our MAC and not Broadcast) ? Then Drop
Not our ethertype of 0x6809? Then Drop
Not a reply? Then Drop
Not our current sequence no? Then Drop
....... (other checks for size,CRC, etc, *could* go here)
Great. Now how do I engineer our Server Code to send a raw frame to test? Linux, makes this fairly easy, and after grokking around some linux PACKET man pages, a generous perusing of Stack Overflow, and some time working with WireShark to confirm the Server is sending data, I successfully sent the Command Frame back to the Client as a reply.
Huzzahs! A successful frame delivery from the Server to the CoCo!
Now I wrote a very simple test client to run on the CoCo side:
It (just) manages to (1) init the NIC card, and (2)send out a test broadcast frame, and (3)quit. I rewrote the core Fuzix device driver and redid (some of it, anyway) in asm. Setting up my linux box to listen for broadcast frames, and running the new test client, I received this on the linux box:
Which perfectly matches the fake frame I slammed together in the CoCo! I can receive packets across the LAN from the CoCo. The Hardware appears to work. Now on to frame reception!
First we must review some of how drivewire works: The Client (presumably a CoCo) sends a command to the server, via a serial connection. There is a presumption of no flow control between the Client and Server. The CoCo is not fast enough to catch an interrupt at 57 kbps, so the CoCo must be listening for incoming data from the server so the size of the response packet (if any) must be known by the CoCo ahead of time. This forces the protocol to be CoCo-Centric, where the CoCo also know exactly how big a response is:
CoCo: gimme sector 42.
Server: here is is: blah blah blah
CoCo: print a byte on lpr
CoCo: write this to sector 42: blah blah blah
Notice not all DW commands from the CoCo expect responses.
The Protocol is really meant for a streaming device, like a RS232 port, UART. Where each byte tumbles in at a time. The standard lib, if you will, for anything in the CoCo World that does Drivewire, is Darren Atkinson's standard dwread.asm and dwwrite.asm, providing for reading and writing chars via different devices (most notably the bitbanger at 57/115 kbps)
Examination of HDBDOS, OS9, Fuzix, sources reveals many commands are cobbled together with multiple calls to dwwrite:
// start of new command
dwwrite( DSK_WRITE );
dwwrite( "%d", 42 );
dwwrite( sector_data );
dwread( §or_buff );
// command done!
Our job is to packetize this information to and from the CoCo. Suppose we make the coco's command, one packet, and the server's reply another. This seems a logical simple, just make every command and reponse their own packets:
CoCo Packet to Server: gimme sector 42.
Server Packet to CoCo: here's sector 42 data.
CoCo Packet to Server: print this byte to lpr.
Great!... So if where given a string of dwwrites from the underlying CoCo OS, how are we supposed to know where the command ends? Bum - Dee - Doo - Dah. Short answer: the dwread/dwwrite code interface doesn't really give us enough information to figure out what constitutes a packet unless the underlying dwread/dwwrite figure it out via the data. Suck Fest! - that would be like implementing a drivewire server right in the underlying code: FAIL! And that will be sucky.
Still sucky, but less so, would be to lightly modify the CoCo OS's to call a "flush" method to declare when a packet is done.
The same is True on the other end: I was hoping just to pump packets to and from the LW Wire server, but this suffers from the same dilemma as the coco client, for the underlying software interface is also a stream (stdin, stdout) :(
Another design idea could be to just create a virtual stream between the CoCo and the Server... But then whe just have some wacky version of TCP. :(
Hum....Let's put off this design decision.... For now we'll start with getting the low levels working: talking to the CoCoNIC card and get a Linux box receiving raw layer 2 frames.
Getting packets from Linux
We'll need to slowly build a server / client from the wire outward to both directions. So we'll start with some OSS code from here to test reception of raw eth frames from the LAN:
I changed the reception MAC filter to ff:ff:ff:ff:ff:ff to be able to look at any Broadcast frames. I compiled and this was the result:
ok... looks right, and you can see a couple of IP packets comming in. This proglet filters by ethertype of 0x0800 (IP), and lets analyze the first packet above:
So all makes sense above. Our server machine can receive ethernet II frames. Next well need something on the CoCo end to push ethernet frames out!
A snapshot of my project in it's end-of-competition state:
Download the link below for a playable disk image.
A week before the Retrochallenge started, I had a brainstorm: A multi-player game for the Tandy Color Computer 2 & 3, using IRC as the messaging service. What a hack! I'm somewhat ashamed of the hackiness of my code. Bad/Inconsistant routine names, cluttered variables, and very few comment for myself to figure things out the next time (next time?!?!). And, amazingly, it worked. My two testers both reiterated "having fun" while playing it. One tester, I quote, said "This is the most I've used my coco in a long time". All three of us lost many times.
Basically, the last month of retro-fun was structured somewhat like this:
1. Ideas, brain storming, general IO/data structures
2. Make a game engine.
3. Merge in IRC code from my CoCo IRC client.
4. Glue it all together with schlocky code.
Surprisingly, there wasn't any horrible problems. Everything just kinda worked (with the normal debugging time). I have stretched a few programming muscles:
* Fixed point math
* Sine Waves! "Hi !"
* IRC protocol
* The horrible wonders of XOR graphics blitting.
* Sharing data structures across a network.
* Learned even more about Drive Wire 4.
In the future I would like to do this:
* Confusicate the IRC coms to prevent spoofing.
* Does this work on a real coco2, as planned? I never did hook up Old Bessie, as I lack the proper cable.
* Allow players to "dig in" to the terrain in effort to help create more valleys.
* Allow player's shields to recharge.
* Each player gets certain amounts of energy to divvy up amongst their cannons, shields, and digging.
* code up In-game messaging.
* Two of the four game testers never could connect, for some reason. Fix.
* Fix all the graphical bugs/state problems.
Bleh... This is a 1/2 hour late, LST. Pencil's down; Time's up kids. Now lest practice the "tornado drill", and hide underneath our desks.
After playing a few rounds of GTW one thing became obvious: 4 or 5 people playing together will obliterate the country-side pretty fast. So yesterday I hacked in a "lift terrain" feature to the game. Basically, every 10 seconds or so, the server will calculate the simplified median height of the playfield, and if it's too low on the screen, then issue a message to the clients to raise the entire playfield (terrain, players, and in-air missiles). Basically, I'm scrolling up.
I played a few test games with a tester, CoCoWOPR, and the scheme worked, fairly well. Every time we would get a little low on terrain, the whole screen would scroll up, and give us more stuff to blow away. The play tests also revealed that this "terrain scolling" tended to "flatten out" the terrain - and those neat little mountains, holes, and valleys then to go away. Maybe giving player the ability to "dig in" to the terrain may be the answer.
I'm also slightly bummed out some of my beta testers can seem to connect. I suspect there is some shenanigans between slightly different flavors of Drivewire, Java, or even the OS they are running Drivewire on :(
Yesterday, I rounded up some CoCo geeks for a real multi-player game. (9:00pm EST GMT -5). Even though only Vincent, "CoCoWOPR", could join. Even though the two other players ended up not being able to join us (see 'bugs', below), the first tests went really well. Several new bugs were found, some where fixed, and many, many, people died. Everyone loses GTW, really.
I distributed some disks to volunteers earlier in the day. I left my CoCo with the game server connected while I was at work - in case anyone got curious. The IRC chat log showed that CoCoWOPR joined sometime and shot a bunch of missiles. After arriving back from work, I heard my CoCo making little explosion sounds... IT WORKS ?!?!
My and CoCoWOPR proceeded to play about 2 dozen games. We both lost many times. I think he had the high score of 150 pts, and one 1 HP left. Then I remembered that I could connect with my real CoCo too. (evil laugh) The game turned to 2 on 1 :) So for a brief moment, we had a 3 player game going.
The lag time was very acceptable.
* There is still quite a few graphical mistakes. XOR missiles always suck for that. I probably should just band-aid it with randomized screen redrawing. Every 20hz just refresh a few tiles on the screen, that should help with the 'graphical shits' as I call them.
* Server Confusion. Me and CoCoWOPR had some problems coordinating who was going to be the server. There's no easy way of coordinating between servers on a asynchronous message system. So I'll continue to ignore this problem and let the player figure it out....(plz: only one server per chat channel!!!)
* Bill and Chris were prevented from joining the IRC/game because of a subtle flaw in how I was assuming DW worked. Bill was kind enough to send me a transcript of his side of the pipe:
client: tcp connect irc.freenode.net 6667<CR>
server: OK connected to irc.freenode.net:6667<CR> <--- Yay! I'm connected, we now speak IRC.
client: NICK DUKE<CR><LF>
server: Fail 010 Unknown API 'NICK'<CR> <---- Huh? this is a DW response, NOT a IRC response.
client: USER DUKE 0 * DUKE<CR><LF>
server: Fail 010 Unknown API 'USER'<CR>
server: OK connected to irc.freenode.net:6667<CR> <-- Huh?... oh... now I'm *really* *really* connected.
server: :leguin.freenode.net NOTICE * :*** Looking up your hostname..<CR><LF>
server: :leguin.freenode.net NOTICE * :*** Checking Ident<CR><LF>
server: :leguin.freenode.net NOTICE * :*** Couldn't look up you hostname<CR><LF>
server: :leguin.freenode.net NOTICE * :*** No Ident response<CR><LF>
server: ERROR :Closing Link: 127.0.0.1 (connection timed out)<CR><LF>
Yeah, DW sends back two connection responses. I think I fixed this problem and this revelation.
* CoCoWOPR got suck in a off-the-bottom-of-the-screen hole. This has happened before, but I thought I killed that bug. I guess the cockroaches really will out last us.
* I can imagine with 4-5 people playing there's going to be a ton of explosion sounds. Each sound takes eats away at handling the player's input. This might become annoying.
* I quickly added setup for a non-artifacting version for CoCoWOPR, but the colors leave a bit to be desired.
* CoCoWOPR mentioned having an in-game chat feature. I agree. Not sure how to do it efficiently on a graphical screen... I may have to resort to the text green-screen. (yuck)
NMG: No More Green Screen. Today I eliminated any ugly trace of the CoCo's nuclear-green text mode, and revised my setup system setup routines to use the 256x192 artifacting mode, as used by the rest of the game, much nicer on the eyes.
I also eliminated *some* of the "missile turds" left over from begone missiles. This was a nice clean up and eliminated 90% of the graphical bugs. I'm happy with the results.
I also fleshed out the menu system with the usual options......
Next is finding someone from across the interwebz to play with me :)
1-10 of 16