syscall_emul: [patch 13/22] add system call retry capability

This changeset adds functionality that allows system calls to retry without
affecting thread context state such as the program counter or register values
for the associated thread context (when system calls return with a retry
fault).

This functionality is needed to solve problems with blocking system calls
in multi-process or multi-threaded simulations where information is passed
between processes/threads. Blocking system calls can cause deadlock because
the simulator itself is single threaded. There is only a single thread
servicing the event queue which can cause deadlock if the thread hits a
blocking system call instruction.

To illustrate the problem, consider two processes using the producer/consumer
sharing model. The processes can use file descriptors and the read and write
calls to pass information to one another. If the consumer calls the blocking
read system call before the producer has produced anything, the call will
block the event queue (while executing the system call instruction) and
deadlock the simulation.

The solution implemented in this changeset is to recognize that the system
calls will block and then generate a special retry fault. The fault will
be sent back up through the function call chain until it is exposed to the
cpu model's pipeline where the fault becomes visible. The fault will trigger
the cpu model to replay the instruction at a future tick where the call has
a chance to succeed without actually going into a blocking state.

In subsequent patches, we recognize that a syscall will block by calling a
non-blocking poll (from inside the system call implementation) and checking
for events. When events show up during the poll, it signifies that the call
would not have blocked and the syscall is allowed to proceed (calling an
underlying host system call if necessary). If no events are returned from the
poll, we generate the fault and try the instruction for the thread context
at a distant tick. Note that retrying every tick is not efficient.

As an aside, the simulator has some multi-threading support for the event
queue, but it is not used by default and needs work. Even if the event queue
was completely multi-threaded, meaning that there is a hardware thread on
the host servicing a single simulator thread contexts with a 1:1 mapping
between them, it's still possible to run into deadlock due to the event queue
barriers on quantum boundaries. The solution of replaying at a later tick
is the simplest solution and solves the problem generally.

x86: Rework opcode parsing to support 3 byte opcodes properly.

Instead of counting the number of opcode bytes in an instruction and recording
each byte before the actual opcode, we can represent the path we took to get to
the actual opcode byte by using a type code. That has a couple of advantages.
First, we can disambiguate the properties of opcodes of the same length which
have different properties. Second, it reduces the amount of data stored in an
ExtMachInst, making them slightly easier/faster to create and process. This
also adds some flexibility as far as how different types of opcodes are
handled, which might come in handy if we decide to support VEX or XOP
instructions.

This change also adds tables to support properly decoding 3 byte opcodes.
Before we would fall off the end of some arrays, on top of the ambiguity
described above.

This change doesn't measureably affect performance on the twolf benchmark.

x86: Implementation of Int3 and Int_Ib in long mode

This is an implementation of the x86 int3 and int immediate
instructions for long mode according to 'AMD64 Programmers Manual
Volume 3'.

scons: Fix warnings issued by clang 3.2svn (XCode 4.6)

This patch fixes the warnings that clang3.2svn emit due to the "-Wall"
flag. There is one case of an uninitialised value in the ARM neon ISA
description, and then a whole range of unused private fields that are
pruned.

CPU: Merge the predecoder and decoder.

These classes are always used together, and merging them will give the ISAs
more flexibility in how they cache things and manage the process.

clang: Fix recently introduced clang compilation errors

This patch makes the code compile with clang 2.9 and 3.0 again by
making two very minor changes. Firt, it maintains a strict typing in
the forward declaration of the BaseCPUParams. Second, it adds a
FullSystemInt flag of the type unsigned int next to the boolean
FullSystem flag. The FullSystemInt variable can be used in
decode-statements (expands to switch statements) in the instruction
decoder.

SE/FS: Use the new FullSystem constant where possible.

X86: Don't use "#if FULL_SYSTEM" in the X86 ISA description.

The decoder now checks the value of FULL_SYSTEM in a switch statement to
decide whether to return a real syscall instruction or one that triggers
syscall emulation (or a panic in FS mode). The switch statement should devolve
into an if, and also should be optimized out since it's based on constant
input.

X86: Make syscalls also serialize after.

X86: Make nop a regular, non-microcoded instruction.

Code in the CPUs that need a nop to carry a fault can't easily deal with a
microcoded nop. This instruction format provides for one that isn't.

X86: Make syscall instructions non-speculative in SE.

copyright: Change HP copyright on x86 code to be more friendly

compile: compile on 32 bit hardware

X86: Decode byte sized singed divide as byte sized.

X86: Use the new forced folding mechanism for the SAHF and LAHF instructions.

X86: Implement far jmp.

X86: Implement the int system call interface in the decoder.

X86: Implement and hook up STI and CLI instructions.

X86: Implement a partial, sort of correct version of the protected mode variant of iret.

X86: First crack at far returns. This is grossly approximate.

X86: Reorganize segmentation and implement segment selector movs.

X86: Implement the PUSHF, POPF, SAHF, and LAHF instructions.

X86: Impelement the HLT instruction and fix the "halt" microop.

X86: Implement the ENTER instruction. This could probably be optimized by cleaning up the indexing in the main loop.

X86: Replace "group10" placeholder with the corresponding instructions in the decoder.

X86: Implement the string IO instructions, ins and outs.

X86: Implement the undocumented SALC instruction which sets AL to 0xFF if CF=1 and 0x00 otherwise.

X86: Implement the XLAT instruction.

X86: Implement the BOUND instruction.

X86: Make "Inst" the default format instead of WarnUnimpl for one byte opcodes.

X86: Implement the in/out instructions. These will still need support from the TLB and memory system.

X86: Implement the LOOP instructions.

X86: Attach the CMC instruction to the decoder.

X86: Implemented the jrcx instruction.

X86: Added some new versions of MOV and a new argument type tag.

X86: Implemented LODS.

X86: Distinguish between the rep and repe prefixes.
STOS and MOVS only accept the rep prefix which always loops until rcx becomes
0. The other string instructions accept repe (same encoding as rep) and repne
which also check the condition code flags each iteration.

X86: Hook in another version of the XCHG instruction.

X86: Implement MOVS

X86: Implement STOS.

X86: Implement idiv and propogate the mul corner case fix.

X86: Implement cmps (string compare)

X86: Implemented and hooked in SCAS (scan string)
Fixed the asz assembler symbol.
Adjusted the condion checks to have appropriate options.
Implemented the SCAS microcode.
Attached SCAS into the decoder.

X86: Implement microops and instructions that manipulate the flags register.

X86: Add decoding for x87 floating point.

X86: Attach the "DIV" instruction implementation to the decoder.

X86: Hook in the new instructions.

X86: Hook in shift and rotate by one instructions, and NOT.

Hook in a bunch of new instructions, fix a few minor bugs, and expand out one of the prefix multiplexed opcode groups.

Implement pusha, popa, three operand imul, hook them into the decoder, and clean up the decoder a little.

Make the operand size reflect the size specifier on the operand tags, and implement NEG

Implemented and hooked in xchg, rotate with carry, and ret instructions

Fixed the distinction between far and near versions of jmp, call and ret. Implemented some shifts, rotates, and pushes.

Implement UD2 and replace the place holder in the decoder.

Implement adc and sbb instructions and microops.

Implement the rest of the conditional jump instructions and hook them into the decoder.

Make the decoder take advantage of the new "B" operand format which takes a register index from the opcode itself.

Define and fill out a lot of different instructions and instruction versions. Added two of the shift microops.

Hook in newly implemented instructions.

x86 fixes
Make the emulation environment consider the rex prefix.
Implement and hook in forms of j, jmp, cmp, syscall, movzx
Added a format for an instruction to carry a call to the SE mode syscalls system
Made memory instructions which refer to the rip do so directly
Made the operand size overridable in the microassembly
Made the "ext" field of register operations 16 bits to hold a sparse encoding of flags to set or conditions to predicate on
Added an explicit "rax" operand for the syscall format
Implemented syscall returns.

Hook near returns into the decoder.

Implement some forms of add.

Fix the operand types in a section of the decoder.

Hook x86 nop into the decoder.

Implemented jnz.

Implement the jz instruction.

Fix a typo in one of the operand type tags.

Make memory instructions work better, add more macroop implementations, add an lea microop, move EmulEnv into it's own .cc and .hh.

More faithfulness to what instructions should work in what modes, and added the MOVSXD instruction.

Make instructions that are illegal in 64 bit mode not do the wrong thing in 64 bit mode. Also add in more versions of PUSH and POP, and a version of near CALL.

Implement a handful more instructions and differentiate macroops based on the operand types they expect.

Partially implement "POP"

Add in MOV instructions.

Fixed format arguments for XOR.

Reworking x86's microcode system. This is a work in progress, and X86 doesn't compile.

src/arch/x86/isa/decoder/one_byte_opcodes.isa:
src/arch/x86/isa/macroop.isa:
src/arch/x86/isa/main.isa:
src/arch/x86/isa/microasm.isa:
src/arch/x86/isa/microops/base.isa:
src/arch/x86/isa/microops/microops.isa:
src/arch/x86/isa/operands.isa:
src/arch/x86/isa/microops/regop.isa:
src/arch/x86/isa/microops/specop.isa:
Reworking x86's microcode system

x86 work that hadn't been checked in.

src/arch/x86/isa/decoder/one_byte_opcodes.isa:
Give the "MOV" instruction the format of it's arguments. This will likely need to be completely overhauled in the near future.
src/arch/x86/predecoder.cc:
src/arch/x86/predecoder.hh:
Make the predecoder explicitly reset itself rather than counting on it happening naturally.
src/arch/x86/predecoder_tables.cc:
Fix the immediate size table
src/arch/x86/regfile.cc:
nextnpc is bogus

Changed some instruction names to be in all caps, and "implemented" move to test the stub code for instructions.

The process of going from an instruction definition to an instruction to be returned by the decoder has been fleshed out more. The following steps describe how an instruction implementation becomes a StaticInst.

1. Microops are created. These are StaticInsts use templates to provide a basic form of polymorphism without having to make the microassembler smarter.
2. An instruction class is created which has a "templated" microcode program as it's docstring. The template parameters are refernced with ^ following by a number.
3. An instruction in the decoder references an instruction template using it's mnemonic. The parameters to it's format end up replacing the placeholders. These parameters describe a source for an operand which could be memory, a register, or an immediate. It it's a register, the register index is used. If it's memory, eventually a load/store will be pre/postpended to the instruction template and it's destination register will be used in place of the ^. If it's an immediate, the immediate is used. Some operand types, specifically those that come from the ModRM byte, need to be decoded further into memory vs. register versions. This is accomplished by making the decode_block text for these instructions another case statement based off ModRM.
4. Once all of the template parameters have been handled, the instruction goes throw the microcode assembler which resolves labels and creates a list of python op objects. If an operand is a register, it uses a % prefix, an immediate uses $, and a label uses @. If the operand is just letters, numbers, and underscores, it can appear immediately after the prefix. If it's not, it can be encolsed in non nested {}s.
5. If there is a single "op" object (which corresponds to a single microop) the decoder is set up to return it directly. If not, a macroop wrapper is created around it.

In the future, I'm considering seperating the operand type specialization from the template substitution step. A problem this introduces is that either the template arguments need to be kept around for the specialization step, or they need to be re-extracted. Re-extraction might be the way to go so that the operand formats can be coded directly into the micro assembler template without having to pass them in as parameters. I don't know if that's actually useful, though.

src/arch/x86/isa/decoder/one_byte_opcodes.isa:
src/arch/x86/isa/microasm.isa:
src/arch/x86/isa/microops/microops.isa:
src/arch/x86/isa/operands.isa:
src/arch/x86/isa/microops/base.isa:
Implemented polymorphic microops and changed around the microcode assembler syntax.

Reworking how x86's isa description works. I'm adopting the following definitions to make figuring out what's what a little easier:

MicroOp: A single operation actually implemented in hardware.
MacroOp: A collection of microops which are executed as a unit.
Instruction: An architected instruction which can be implemented with a macroop or a microop.

A batch of changes and fixes. Macroops are now generated automatically, multiops do alot more of what they're supposed to (excluding memory operands), and microops are slightly more implemented.

Made the MultiOp format do a little more. It now sets up single microop instructions to return an instance of the right class. The code to decode register numbers and generate loads and stores still needs to be added. Also, a syntax for specifying operands as sources, destinations, or both needs to be established. Multipl microop instructions are also not handled, pending real macroop generation support.

Fidget with the syntax of the MultiOp format in anticipation of making it actually work.

Start implementing groups of instructions which do the same thing on different sets of inputs.

Break out the one and two byte opcodes into different files. Also change what bits decode is done on to reflect where clumps of instructions are.