BiDiB.org

Communication

Summary

Serial connection

Specification

BiDiBus

Network

Specification
Background information

Background Information on netBiDiB

The protocol specification for netBiDiB evolved from consultation between control software developers, node engineers and end users, considering security, safety, usability, functionality and implementation complexity. On this page, the design decisions are illuminated and a closer look on the functionality of netBiDiB is given in the form of an FAQ.

Why are both UDP and TCP utilised?

The aim of BiDiB is the realisation of a faultless communication with the model railway. This is important not only for a reliable normal operation but also for firmware updates. UDP does not guarantee delivery or packet order, which causes a high error rate especially over radio service via Wifi.

Though BiDiB already has end-to-end validation through the message sequence numbers and important messages are protected by automatic repeat when missing acknowledgement, those measures are designed for infrequent disturbances and not error-prone transport channels. Hence an additional safety layer on the individual data link is necessary and also lessens the effort necessary for an application-specific protection.

On these grounds we chose TCP for the communication, despite its higher memory and processing power requirements affecting smaller devices. On the other hand TCP does not offer broadcast, which is required for the user-friendly automatic discovery of devices.

Thus UDP is used for the discovery step, the remaining communication is conducted via TCP.

Why was encryption waived?

Any communication in the internet (especially when enabling remote firmware updates) must be encrypted from today's perspective. However a secure implementation incurs considerable expenses and requires a comprehensive security concept that represents a limitation in many areas when covering all aspects.

A model railway control is usually operated in a secured local home network, there is no data traffic over the internet. Where this is essential, a virtual private network (VPN) can offer the necessary protection.

Even when for the sake of convenience currently no encryption is mandated, the architecture is already structured in a way to be usable with secured connections.

Why is Pairing necessary?

A netBiDiB system should be able to connect all by itself, but only amongst those devices the user wants. This should work even when there are multiple independent netBiDiB systems in the same network, which may happen e.g. on enthusiast meetings or in clubs with multiple (parts of) layouts.

Who shall now connect whereto? With the pairing concept we created a comfortable way to keep together one's 'own' layout part in such environments as well, and at the same time allow automatic connection establishment.

Why does also the host need an UID for Pairing?

During the pairing not only the host remembers the connected nodes, but also the nodes themselves remember their preferred connection partner. This does i.a. prevent that an off-the-shelf host program can be used for a 'hostile takeover' of a system - neither inadvertently nor deliberately. The various program instances of course need a distinguishing feature, the utilisation of the Unique-ID infrastructure for this seemed natural.

How to chose the UID of the host?

The Unique-ID of a host program instance should preferably globally unique, but this cannot be implemented without a central registry. Such an infrastructure is however unreasonably expensive, where it is not already in place (e.g. for awarding program licences) a random generator may be employed. A randomly generated serial number is stored either with the program settings or in the layout plan file, so that it does not change with every program start. Alternatively the serial number may be produced via a hash function from a characteristic value of the computer (e.g. the MAC address).

As with nodes, the Unique-ID begins with a vendor ID, every vendor may freely dispose of his number space and choose an appropriate assignment method. Program vendors without an NMRA identifier may obtain a 16-bit product identifier from the range for open source components (VID 13).

How to chose the name in the descriptor of the host?

The descriptor serves the user to easily distinguish multiple connections instead of having only the UIDs for comparison. On a device with GUI, e.g. a throttle or a config tool, the user may select on which of multiple available hosts the node should log on. The product and participant name should therefore be as significant as possible.

For the product name the program name should be used, the participant name may either be directly input by the user or be generated automatically from the computer name, operating system login or the name of he opened plan file.

The Pairing process is much too complicated.

The status of the link can be modelled by a simple asynchronous state automaton. There are only 5 states:

none: the link has not yet been initialised, the remote end is unknown
unpaired: both parties do not trust each other (or don't know about it)
their-accept: the remote end trusts the participant, but not in reverse
my-accept: the participant trusts the remote end, but not in reverse (or it is still unknown, waiting for a response)
paired: both link partners trust each other

The pairing procedure is controlled by a second asynchronous state automaton with only 3 states:

idle: currently no pairing process takes place
my-request: the participant has sent a request and waits for the handshake reply (or for the end of the pairing process)
their-request: the participant has received a request and waits for the user (or for the end of the pairing process)

A pairing process ends with reaching the link status paired, on expiration of the timer, on rejection/cancellation by the user, or on rejection/cancellation by the remote end (unsolicited STATUS_PAIRED/STATUS_UNPAIRED).


  State transitions
  
    Event Reaction for link Reaction for pairing process
  
  
    
      TCP-ACK
      other := null
link := none
send(DESCRIPTOR_UID, my_uid)
      
    
    
      TCP-FIN
      other := null
link := none
      pairing := idle
    
    
      receive(DESCRIPTOR_UID, their_uid)
      other := their_uid
if known_trusted(other):
  send(STATUS_PAIRED)
  link := my-accept
else:
  send(STATUS_UNPAIRED)
  link := unpaired
      
    
    
      on_paired()
      store_trust(other)
      pairing := idle
    
    
      receive(STATUS_PAIRED)
      if link == unpaired:
  link := their-accept
if link == my-accept:
  on_paired()
  link := paired
      
    
    
      receive(STATUS_UNPAIRED)
      if link != none:
  remove_trust(other)
  link := unpaired
      pairing := idle
    
    
      on_requests_exchange_complete()
      if link != none:
  send(STATUS_PAIRED)
if link == unpaired:
  link := my-accept
if link == their-accept:
  on_paired()
  link := paired
      
    
    
      receive(PAIRING_REQUEST)
      
      if pairing == my-request:
  on_requests_exchange_complete()
if pairing == idle:
  if link != none:
    pairing := their-request
    
    
      User initiates pairing
      
      if pairing == idle:
  send(PAIRING_REQUEST)
  pairing := my-request
    
    
      User accepts pairing
      
      if pairing == their-request:
  send(PAIRING_REQUEST)
  on_requests_exchange_complete()
    
    
      User denies pairing
or revokes it
      if link != none:
  remove_trust(other)
  send(STATUS_UNPAIRED)
  link := unpaired
      pairing := idle
    
    
      User (or timeout)
cancels pairing
      if known_trusted(other):
  send(STATUS_PAIRED)
else:
  send(STATUS_UNPAIRED)
      pairing := idle

State transitions
Event	Reaction for link	Reaction for pairing process
TCP-ACK	other := null link := none send(DESCRIPTOR_UID, my_uid)
TCP-FIN	other := null link := none	pairing := idle
receive(DESCRIPTOR_UID, their_uid)	other := their_uid if known_trusted(other): send(STATUS_PAIRED) link := my-accept else: send(STATUS_UNPAIRED) link := unpaired
on_paired()	store_trust(other)	pairing := idle
receive(STATUS_PAIRED)	if link == unpaired: link := their-accept if link == my-accept: on_paired() link := paired
receive(STATUS_UNPAIRED)	if link != none: remove_trust(other) link := unpaired	pairing := idle
on_requests_exchange_complete()	if link != none: send(STATUS_PAIRED) if link == unpaired: link := my-accept if link == their-accept: on_paired() link := paired
receive(PAIRING_REQUEST)		if pairing == my-request: on_requests_exchange_complete() if pairing == idle: if link != none: pairing := their-request
User initiates pairing		if pairing == idle: send(PAIRING_REQUEST) pairing := my-request
User accepts pairing		if pairing == their-request: send(PAIRING_REQUEST) on_requests_exchange_complete()
User denies pairing or revokes it	if link != none: remove_trust(other) send(STATUS_UNPAIRED) link := unpaired	pairing := idle
User (or timeout) cancels pairing	if known_trusted(other): send(STATUS_PAIRED) else: send(STATUS_UNPAIRED)	pairing := idle

State diagrams with the transitions of both automatons. Drawn with dashed lines: transitions from any state.

Small deviations from this standard model are possible in more sophisticated implementations:

When receiving a STATUS_UNPAIRED, remove_trust(other) need not be called if there is a way for the user to revoke an individual pairing on the device. The link state may then transition from paired/my-accept to my-accept (instead of all the way to unpaired).
When revoking a pairing (by sending STATUS_UNPAIRED), the link state may transition from paired to their-accept (instead of all the way to unpaired).
When receiving a PAIRING_REQUEST in the states paired or my-accept, STATUS_PAIRED may be sent immediately (instead of only after the user accepts the request or the timeout).
When receiving a PAIRING_REQUEST in the state paired, the pairing process may remain idle (and not show a dialog to the user, just immediately respond with STATUS_PAIRED).
When the user accepts a pairing request (in their-request), the PAIRING_REQUEST and STATUS_PAIRED messages may be sent in any order.
It is assumed that a user may (only) initiate a pairing from the states unpaired, my-accept and their-accept. To implement a "pairing mode" that could be entered even from the state none or before the TCP connection is opened, the PAIRING_REQUEST must not be sent before the DESCRIPTOR_UID is received and unpaired or my-accept has been reached.
When the PAIRING_REQUEST-handshake is completed in the states my-request/their-request, the process automaton may not stay in the respective state but transition to an intermediate fourth state both-requests, from which e.g. the user can not accept the request again.

Is netBiDiB a 1:1 or 1:N connection?

Both use cases are covered by netBiDiB. In principle:

a client decides to how many servers it establishes a link
a server decides from how many clients it accepts links (concurrently)
a node decides on which interface to logon (maximum one)
a interface decides how many (concurrent) logons of nodes it accepts (maximum 255)

A server shall manage as many links as his resources admit. A client (interface, host) can decide whether to utilise only single connections or arbitrarily many. From the user perspective, having the potential for multiple connection is preferable, especially when implementing discovery this is inevitable for reading in the descriptors of all links.

A user interface of a host program that enables multiple connections may for instance look like this:

How can a node switch between hosts?

On a physical bus, the interface at which a node should logon can easily be selected by the plug connection and a change can be performed through simple re-plugging. For a purely virtual bus this 'unplugging' needs to be modelled explicitly in the software though.

The termination of the TCP connection would be one possibility, but prevents are future reconnection by the node. Sending a MSG_LOCAL_LINK STATUS_UNPAIRED would work as well, but implies a trust revocation and prevents a new logon without prior pairing.

Therefore only the logon registration is suspended, this may be carried out either by the node via MSG_LOCAL_LOGOFF or by the interface via MSG_LOCAL_LOGON_REJECTED. The link stays open (paired) and is available for a re-logon.

The logged-off node now automatically selects a different interface to which it is paired (i.e. which is connected and trusted), and tries to logon there. The user can thereby easily let a node alternate between multiple programs that all have established a link with the node, just be releasing the connection at any one time.

The node should favour those interfaces to which it was connected at last, so that switching back and forth between two programs needs only a single button press.

To allow arbitrary switchover between more than two programs, a PAIRING_REQUEST is understood as a logon invitation. That way, a node immediately connects to an interface after their initial pairing, and a PAIRING_REQUEST on an already paired link (which is instantly acknowledged with STATUS_PAIRED) grants the respective interface the highest priority at the next logon. As soon as the existing logon is released, the node switches to the prioritised interface.