Posts
All Comments
Code the Specification:
6303 Assembler in 2000 Lines of Tcl Using Regular Expressions
I have been interfacing to the Psion Organiser 2 recently, firstly using a Raspberry Pi RP2040 to replace a datapack and provide SD card storage of pack images:
and also in the top slot of the organiser, which provides a way to add hardware to the organiser:
When adding hardware in this way the organiser allows the code that drives the hardware to be held on the hardware as an embedded datapack. This means that you never lose the drivers for a piece of hardware as it is built in to the device itself. I made a prototyping board using a raspberry Pi Pico running modified datapack code from the RP2040 project. The drivers for the hardware are usually written in assembly code as direct access to the hardware is required.
Datapacks that drive hardware usually add commands to the main menu of the organiser, and I wanted to do that for my prototyping hardware. To do this requires building a datapack image containing the command code plus the extra information that datapacks contain that enables the organiser to load the code.
Psion supplied an assembler in the 80s that was able to perform the assembly of code together with the extra features that provide the datapack image. This is still available but only runs under DOS, so must be run in some kind of emulator, or on a native DOS machine. I don't have a DOS machine and although I do have a DOS emulator and have run this assembler, I really wanted a version of the assembler that runs on Linux. There are a few 6303 assemblers about, but I couldn't find one that did all of the special organiser things, so I decided to write one.
An assembler is quite simple in that it takes assembly language instructions from a text file and builds binary object code. The datasheet for a processor usually provides a list of the instructions and the corresponding object code, so lends itself well to a method I call 'Code the Spec'.
The idea of coding the specification is to take some core information directly from the specification, copy it (cut and paste ideally) and build a data structure that the code uses to do whatever it needs to do. In this case the assembly language instructions are copied from the processor datasheet and form an array of data that the code uses to parse input files.
The 6303 datasheet I found has a list like this:
The assembly instructions are in the second column and the machine code is in a column per addressing mode. Addressing modes pose an extra problem as you have to recognise which addressing mode the instruction is using in order to determine the machine code that must be generated. I decided to use regular expressions to parse the assembly instructions as they are compact and powerful and can handle this parsing. The corresponding table in the code for the above instructions is here:
{ADCA ____ 89 99 A9 B9 ____ ____ ____}
{ADCB ____ C9 D9 E9 F9 ____ ____ ____}
{ADDA ____ 8B 9B AB BB ____ ____ ____}
{ADDB ____ CB DB EB FB ____ ____ ____}
{ADDD ____ C3.3 D3 E3 F3 ____ ____ ____}
{AIM ____ ____ ____ ____ ____ ____ 71 61 }
Well, most of the instructions are there, as the datasheet is a scan I couldn't cut and paste from it so I used an alphabetical list of 6303 instructions to create the array. Taking the ADCB instruction as an example:
{ADCB ____ C9 D9 E9 F9 ____ ____ ____}
the instruction to be matched is shown as ADCB and the opcode for each addressing mode is in a column in the same order as the datasheet. You can see that a cut and paste, if you could do it, would quickly create this table. The instruction length for each instruction is set as a default for each addressing mode as that usually fixes the length. there are always some exceptions, and the ADDD instruction has an instruction length over-ride by using C3.3 to specify a length of 3 bytes for this instruction.
The datasheet only shows the addressing modes for the instructions it is describing, so the array has more columns than the datasheet as it has all possible addressing modes.
Each addressing mode modifies the syntax of the instructions, so a list of regular expressions is used to both determine the addressing mode being used, and pull out the operands for each instruction.
these variables help with expression complexity:
set ::RE_EXPR "\[A-Z0-9a-z_$^\]+"
set ::RE_EXPR ".+"
set ::IMM_RE "^%s\[ \t\]+#\[ \t\]*(\[A-Z0-9a-z_$^\]+)"
set ::ADDMODE {
{"REL" 2 "^%s\[\t\]+($::RE_EXPR)"}
{"IMM" 2 "^%s\[ \t\]+#\[ \t\]*($::RE_EXPR)"}
{"DIR" 2 "^%s\[ \t\]+($::RE_EXPR)"}
{"IDX" 2 "^%s\[ \t\]+($::RE_EXPR)\[ \t\]*,\[ \t\]*(\[Xx\])"}
{"EXT" 3 "^%s\[ \t\]+($::RE_EXPR)"}
{"IMP" 1 "^%s\[ \t\]*$"}
{"XIM" 3 "^%s\[ \t\]+#\[ \t\]*($::RE_EXPR)\[ \t\]*,\[ \t\]*($::RE_EXPR)" }
{"XXM" 3 "^%s\[ \t\]+#\[ \t\]*($::RE_EXPR)\[ \t\]*,\[ \t\]*($::RE_EXPR)\[ \t\]*,\[ \t\]*\[Xx\]"}
}
Each addressing mode has the name of th emode, the default instruction length for instructions of that addressing mode and the regular expression that can be used to parse the data and also to decide if an instruction is using a particular addressing mode.
The assember can test each assembly instruction against a regular expression made up of the instruction name and then each addressing mode in turn. When it gets a match it can build the object code from the table entries.
The assembler is a multi pass assembler as the addressing mode
There are some complications, such as macros and code overlays. There are also various directives to define byte, word and string data, and the concept of 'pack address' which is the address that a byte is located at in the 'EPROM' of a datapack, which is different to the address defined using ORG directives. The Psion organiser also requires relocatable code which is done using a 'fixup' list at the end of the object code.
The assembler supports all of these features in just under 2000 lines, and manages to assemble Psion example code with almost exactly the same object code. I say almost exactly as I found that the Psion assembler example I was using used a less efficient addressing mode for one of the instructions, but only one occurance of it. My assembler generated a more efficient equivalent instruction and so was one byte shorter than the Psion version.
The Psion assembler list output (this is from the XDICT.SRC example):
165 20B4 97 E2 sta a,flag:
372 21F2 B7 00E2 sta a,flag
My assembler outputs the first form once it knows that the data is zero-page. The colon at the end of the label may be a reson but I haven't found documentation of that syntax.
The assembler can also embed the object code in a C program as an array using embedded comments to mark where the data should appear. This is useful as the code needs to be in a C file where it is compiled and flashed to a RP Pico.
This example of 'Code the Spec' didn't use exactly the same syntax as the datasheet, for another example, have a look at the Z80 Arduino Shield
| "00rrr110 nn :LD r, n:", |
| "01rrr110 :LD r, (HL):", |
| "11y11101 01rrr110 dd :LD r, (y+d):", |
| "01110rrr :LD (HL), r:", |
| "11y11101 01110rrr dd :LD (y+d), r:", |
| "01rrrsss :LD r,s:", |
If you look in the Z80 programming databook you will see that the instructions are defined in exactly this form, and the table was copied from those definitions.
Finite State Machines and the Software Hammer
The firmware that runs the Casio FX502P gadget that I recently created has a structure and uses a technique that I though was worth documenting. The gadget uses the Arduino framework and is described in the following videos and blog entry.
This device has some tight timing constraints while performing interactions with a real-time interface. It cannot waste time deciding what to do next as it has a bidirectional synchronous bit stream to attend to at about 200kHz. The processor itself runs at tens of MHz so there's time for several instructions to run between clock edges, but not enough time that the code can perform time consuming operations.
As the protocol isn't officially documented I'd like to be able to change the interface behaviour to add new commands, remove commands and maybe alter some of the existing commands. The code needs to be easy to understand and change.
So, we have two requirements:
A. Code must run fast
B. Code must be easy to understand and not break if altered
These requirements, even if they don't officially declare war on each other, fight against one another. Code that is easy to understand and change isn't usually fast, and code that is fast isn't usually easy to understand or change.
Fast Code
One fast way to respond to inputs is to use interrupts. Something happens and a fragment of code runs. That's the basis of interrupts. If you have something simple then it's relatively easy to implement. A push-button, for instance, is a single input and an interrupt service routine (ISR) that runs when the input is either changed or at a certain logic level. (An interrupt that runs code when an input changes is edge-triggered, one that fires at a certain level is, well, level-triggered).
In the push-button example it's all straightforward. The push-button is pressed, the input changes level and the code runs. Whatever is needed to be done is done and the ISR exits. There are details that have to be attended to, such as using the volatile keyword (in C), we are more interested in the overall structure here than details.
For the FX502P gadget the interface has:
A clock line that runs at approximately 200kHz and clocks data on both edges
A data line, which is bidirectional
Two control lines, active high
A serial data format that uses packets of different lengths, including a longer packet that has a payload with start bit, stop bits,data and parity, which also gets clocked out on a different edge to the data clocked in.
We therefore have edge triggered signals, level triggered signals and have to collect and build packets of different lengths and formats. Suddenly it's all a lot more complicated than a push-button.
At the time of writing (and as this code is easy to change, it changes now and again), there are two interrupts, one driven from the clock and one from a chip enable (CE) control line. The complexity has just moved up a notch. There's now two interrupts that have to be handled. Fortunately in this case they cannot execute simultaneously, although the method I have used doesn't have problems if they do execute simultaneously (on different cores, for example).
Orchestration
So how can you create something that can react to interrupts and perform actions based on those interrupts? Not only react, but react in a way that has memory of what has happened in the past. For instance, if the clock changes state do we clock data in or out? We need to know if we are in the middle of receiving a packet or sending one.
In the gadget this is done with a Finite State Machine (FSM). It's a thing (Machine) that can be in one of a Finite number of States at any time. FSMs move between states when they receive an input (stimulus). (An FSM is always in a state, and changing from one state to another logically takes no time). There are many ways an FSM can drive outputs, in the one used here, a function is called when the FSMmoves to a new state. An action is taken on entry to a state.
When an ISR runs, it sends a stimulus to the FSM and that may causes a state change. Code is run when entering a new state, that code can do anything required at that time.
Implementation
There are several ways to implement an FSM. As the gadget doesn't have time to spare it uses nested switch statements. You can use a table driven approach but searching the table uses processor cycles and hence more time than a switch statement approach. Using a table, though, allows you to use a higher level of abstraction when defining the FSM. With the gadget I wanted a similar abstraction, but to achieve this I used what I call a 'software hammer'. The idea is to have an abstract description of what you want to achieve, in a form that is easy to understand and easy to alter, and support code that hides the details. This support code is 'hammered' into shape to support the abstract code and handle all the details.
In the gadget code, the nested switch FSM is the abstract part, the rest of the code is hammered to support that abstraction.
Here's a fragment of the FSM code which shows the two nested switch statements:
sets up the length of the packet.
Then:
isr_send_flag = true;
indicates to the support code that a transmission is needed.
There's some other housekeeping stuff, because the universe requires it, such as the SET_DATA_BIT0 which is exposed here due to the timing of the interface. It can't be done later.
What is important is that the fact that the received data packets are clocked in on one edge of the clock while the transmitted packets are clocked out on the other edge is completely hidden at this layer of abstraction. That detail has been hammered into the support code.
Even though the details of the support code are probably horrible with lots of special case required by the interface, that doesn't matter once it works, as from then on it should only be necessary to change the abstract code when new packets need to be decoded or transmitted. The support code handles the interface, and that isn't going to change.
The whole of the FSM looks pretty much like these two pieces of code, although there are other things that need to be done, like this:
Notes
FSM State
The state an FSM is in is determined by it's state variable. Whatever number is in that variable is the state it is in. It can't be in two states at once and it take no time (logically) to update a variable's value.
DFSM
These FSMs are actually DFSMs (Deterministic Finite State Machines), which means that the state transitions are deterministic. Non deterministic state machines can also be useful, their state transitions can be random in some way. They are not that common, though.
Race Conditions
FSMs can be resilient against race conditions. If stimulii are queued and then processed then you can build your FSM to handle stimulii in any order and still get the correct behaviour. Every state should be examined to see what action it should take for every possible stimulus. It doesn't matter when or in what order the stimulii arrive, the FSM states will always take the appropriate action, and it won't miss stimulii as long as the queuing code is correct.
Posts
All Comments
Posts
All Comments
