SpaceFibre

TEMU provides support for SpaceFibre based devices. SpaceFibre is a successor to SpaceWire, and parts of the emulator infrastructure builds on top of it (e.g. RMAP command interpretation, which are encoded identically). The port interface is available in: temu-c/Bus/SpaceFibre.h.

SpaceFibre is a point to point bus. Two devices can be connected to each other directly. Routers can be used to connect multiple devices.

On a packet level, the biggest difference between SpaceWire and SpaceFibre is that SpaceFibre has virtual channels (up to 256 per link). The SpaceWire interface documentation goes into more detail on how routing works.

Interfaces

The interface temu_SpfiPortIface is defined in the temu-c/Bus/SpaceFibre.h header.

SpaceFibre Interface

typedef enum {
  teSPFILS_Illegal,
  teSPFILS_Offline,
  teSPFILS_Init,
  teSPFILS_StartUp,
  teSPFILS_Ready,
  teSPFILS_Run,
} temu_SpfiLinkState;

typedef enum {
  teSPFIFL_DataPacket = (1 << 0),
  teSPFIFL_Broadcast = (1 << 1),
  teSPFIFL_ErrorEndOfPacket = (1 << 2),
  teSPFIFL_TimeDistribution = (1 << 3),
  teSPFIFL_FlowControl = (1 << 4),
  teSPFIFL_DataCRCError = (1 << 5),
  teSPFIFL_HeaderCRCError = (1 << 6),
  teSPFIFL_SeqNumWrong = (1 << 7),
  teSPFIFL_Full = (1 << 8),
  teSPFIFL_Retry = (1 << 9),
  teSPFIFL_ACK = (1 << 10),
  teSPFIFL_NACK = (1 << 11),
  teSPFIFL_BroadcastLate = (1 << 12),
  teSPFIFL_BroadcastDelayed = (1 << 13),
} temu_SpfiFlags;

typedef struct temu_SpfiPacket {
  uint32_t Flags;
  uint8_t Channel;
  uint8_t SeqNum;
  uint8_t Type;
  temu_Buff PktData;
} temu_SpfiPacket;

typedef struct temu_SpfiControlWord {
  uint32_t Flags;
  uint8_t Channel;
  uint8_t SeqNum;
  uint8_t Value;
} temu_SpfiControlWord;

struct temu_SpfiPortIface {
  void (*connect)(temu_Object *Dev, temu_SpfiPortIfaceRef OtherSide);
  void (*disconnect)(temu_Object *Dev);
  void (*signalLinkStateChange)(temu_Object *Dev, temu_SpfiLinkState LinkState);
  temu_SpfiLinkState (*getOtherSideLinkState)(temu_Object *Dev);
  void (*sendControlWord)(temu_Object *Dev, temu_Object *Sender,
                          temu_SpfiControlWord *ControlWord);
  void (*sendPacket)(temu_Object *Dev, temu_Object *Sender,
                     temu_SpfiPacket *Pkt);
};

While the SpaceFibre protocol is character based, supports multiple lanes per link, and may split packets into multiple frames for QoS, TEMU transfers full packets at a time for performance reasons. The temu_SpfiPacket type is used for broadcast messages as well (most often used for time codes). Control Words (such as Flow Control Tokens) are modeled in the interface and can be represented, but not used by any device model shipped as part of TEMU 5.0.0 at the moment.

The SpaceFibre packet structure is used to pass a packet between nodes. The Flags field will mark it as a broadcast or normal data packet. Additional flags can be present to indicate e.g. a error end of packet. Sequence numbers in SpaceFibre are assigned per virtual channel frame, and not per packet. They are only exposed in the interface to allow for error injection in the future (or by user defined devices) and currently unused. As for the SpaceWire packet, a reference-counted TEMU buffer is used to store the payload. Functions defined in temu-c/Bus/Spacewire.h can be used to decode RMAP commands transfered via SpaceFibre packets.

SpaceFibre links are full-duplex. A link (no matter how many lanes) is modeled by having both sides implement the temu_SpfiPortIface interface and holding a reference to the other side.

For link initialization, the temu_SpfiLinkState enum and signalLinkStateChange / getOtherSideLinkState functions are used instead of control words. Idle frames are not modeled either.

Flow Control

According to the SpaceFibre protocol, every device/link must send flow control tokens to its peer, indicating how many bytes are guaranteed to be accepted per VC. A sender should not send any data for a VC if it did not receive flow control tokens first. Flow control tokens flow from the receiver to the sender! This can be modeled using the interface. The Value field of the temu_SpfiControlWord structure contains the multiplier value contained in flow control tokens.

The models currently shipped with TEMU 5.0.0 do not do this! They will ignore received FTCs and never send ones out themselves.

Fault Injection

SpaceFibre supports fault injection via custom man-in-the-middle models. These models can inject packets using events or modify packets in-flight.

For example, a mismatch in the SpaceFibre control word CRC values can be emulated using the teSPFIFL_DataCRCError flag. The CRC value itself that is part of the SpaceFibre frame end control word is not modeled.

Since the custom model acts as man-in-the-middle, it can also drop packets in transit. This could then be detected by the other side based on the sequence numbers.

Examples

The example below shows how to use the SpaceFibre interface.

The example below sends a response without checking how many and what kind of flow control tokens have been received.

SpaceFibre Device Example

#include <stdint.h>
#include <stdio.h>
#include <string.h>
#include <assert.h>

#include "temu-c/Support/Objsys.h"
#include "temu-c/Support/Attributes.h"
#include "temu-c/Support/Logging.h"
#include "temu-c/Bus/SpaceFibre.h"

typedef struct {
  temu_Object Super;

  temu_SpwLinkState LinkState;
  temu_SpfiPortIfaceRef Uplink;
} SpfiDevice;

static void*
create(const char *Name,
       int Argc TEMU_UNUSED,
       const temu_CreateArg *Argv TEMU_UNUSED)
{
  SpfiDevice *Dev = malloc(sizeof(SpfiDevice));
  memset(Dev, 0, sizeof(SpfiDevice));
  Dev->LinkState = teSPFILS_Offline;
  return Dev;
}

static void
destroy(void *Obj)
{
  free(Obj);
}

static void
connect(temu_Object *Obj, temu_SpfiPortIfaceRef OtherSide)
{
  SpfiDevice *Dev = (SpfiDevice*)Obj;
  Dev->Uplink = OtherSide;
}

static void
disconnect(temu_Object *Obj)
{
  SpfiDevice *Dev = (SpfiDevice*)Obj;
  Dev->Uplink = {nullptr, nullptr};
}

static void
linkStateChangeSignaled(temu_Object *Obj, temu_SpfiLinkState LinkState)
{
  SpfiDevice *Dev = (SpfiDevice*)Obj;
  if (Dev->LinkState == teSPFILS_Offline && LinkState == teSPFILS_Init) {
    Dev->LinkState = teSPFILS_Init;
  } else if (Dev->LinkState == teSPFILS_Init && LinkState == teSPFILS_StartUp) {
    Dev->LinkState = teSPFILS_StartUp;
  } else if (Dev->LinkState == teSPFILS_StartUp && LinkState == teSPFILS_Ready) {
    Dev->LinkState = teSPFILS_Ready;
  } else if (Dev->LinkState == teSPFILS_Ready && LinkState == teSPFILS_Run) {
    Dev->LinkState = teSPFILS_Run;
  } else {
    temu_logSimError(Dev, "Link state transition unhandled");
    return;
  }

  Dev->Uplink.Iface->signalLinkStateChange(Dev->Uplink.Obj, Dev->LinkState);
}

static temu_SpfiLinkState
getLinkState(temu_Object *Obj)
{
  SpfiDevice *Dev = (SpfiDevice*)Obj;
  return Dev->LinkState;
}

static void
receiveControlWord(temu_Object *Obj, temu_Object *Sender,
                temu_SpfiControlWord *ControlWord)
{
  SpfiDevice *Dev = (SpfiDevice*)Obj;
  temu_logInfo(Dev, "Received SpaceFibre control word...");

  // Otherwise unused...
}

static void
receivePacket(temu_Object *Obj, temu_Object *Sender, temu_SpfiPacket *Pkt)
{
  SpfiDevice *Dev = (SpfiDevice*)Obj;

  // NOTE: A real model should avoid calling back into the other side directly!
  // It should use events to delay the response!
  char Buf[1024];
  size_t Len = snprintf(&Buf, sizeof(Buf),
      "Hello! I received you packet on virtual channel %d of %d bytes.",
      Pkt->Channel, temu_buffLen(&Pkt->PktData));

  temu_SpfiPacket Response = {
    .Flags = teSPFIFL_DataPacket,
    .Channel = Pkt->Channel,
    .PktData = temu_buffCreate(Len)
  };
  memcpy(temu_buffReadableData(&Response.PktData), Buf, Len);
  Dev->Uplink.Iface->sendPacket(Dev->Uplink.Obj, &Dev.Super, &Response);
  temu_buffDispose(Response.PktData);
}

TEMU_PLUGIN_INIT
{
  temu_Class *C = temu_registerClass("SpfiDevice", create, destroy);

  temu_addInterfaceReference(C, "SpfiPort", offsetof(SpfiDevice, Uplink),
                             TEMU_SPFI_PORT_IFACE_TYPE, 1, 0, nullptr, nullptr,
                             "SpaceFibre port to the other side of the link.");

  static temu_SpfiPortIface SpfiPortIface = {
    .connect = connect,
    .disconnect = disconnect,
    .signalLinkStateChange = linkStateChangeSignaled,
    .getOtherSideLinkState = getLinkState,
    .sendControlWord = receiveControlWord,
    .receivePacket = receivePacket
  };
  temu_addInterface(C, "SpfiPortIface", TEMU_SPFI_PORT_IFACE_TYPE,
                    &SpfiPortIface, 0, "SpaceFibre port interface");
}