Support for BiDiB

FAQ

Questions and Answers for the implementation of BiDiB has been collected here:

I want to implement BiDiB in my host program, what things do I have to integrate?

The feature coverage that a program implements is essentially at the discretion of the programmer. It is for example possible that a configuration manager focuses on ports, macros and firmware updates, while in contrast an operation application uses feedback, driving commands and accessory controls without caring about details of node configurations.

The only mandatory parts are the safeguards for securing data transmission and the basic functions for reading in and displaying nodes.

How do I deal with the various protocol versions in my program?

BiDiB is constructed so that full backward compatibility is possible. For this purpose, reading in any node should begin with MSG_SYS_GET_P_VERSION and MSG_SYS_GET_MAGIC. It is discernible from that information which messages the node supports and which it does not support (any more), as well as which state model it uses and which features need to be queried.

How to deal with this depends completely on the software, bearing in mind the different release cycles of the components. Old protocol version should be supported as far as possible by means of a compatibility layer. If that is not possible or too laborious, the limited compatibility must be documented in the manual.

When an outdated or unheard-of new protocol version which is not or not completely supported is detected, a warning (about restricted functionality) or error message ("please update") is indicated. Precisely for forward compatibility it is important to keep up the functionality and usability of the software. Unknown message types, message parameters, features, configurations enums etc. should simply be ignored.

Is there an overview for programs that shows how data is transferred and with which interface parameters?

The protocol distinguishes between the message content (payload) and transportation route. The available transportation routes and their data protection mechanisms are described in the 'Transport' section of this web page.

Currently (as in 2016) there are three gateways with BiDiBus interfaces in the market: Tams S88-BiDiB, GBMBoost and BiDiB-IF2, all of them are based on USB with a virtual serial port.

Where can I find the exact definition of the message numbers?
There is a mandatory header file, where the message numbers are defined. Only this header file should be used in programs and the message number should be inserted via #define. This header file can be found in the file area at BiDiB-Implementation-Group.
I have a BiDiB system and have a successful connection to the first module. How can I connect to the second module?

If the first module is a hub/interface, then you can query the current assignment of all connected nodes to their local addresses form this first module with: MSG_NODETAB_GETALL. This query gives a list of the connected modules with their UNIQUE_ID and the currently assigned local address. Nodes will be addressed via the local address as follows:

  • The interface will be addressed with 00.
  • The node with the local address 1 will be addressed with 01 00.
  • The node with the local address 2 will be addressed with 02 00.

Suppose there is another hub at the local address 2:

  • The node with the local address 5 behind this hub will be addressed with 02 05 00.

Caution: The list might be different after power off/on (depending on the logon). There is always the need to go via the UNIQUE_ID. A list of possible UNIQUEs as an XML file is available on request.

Was does 'broadcast' mean and which messages are affected?

For some state transitions and messages it is very important that all nodes in the system get informed, i.e. clock messages. Therefore such a message is only sent to the first interface, the interface forwards this message to all subnodes.

The following messages are sent as broadcast:

  • MSG_SYS_ENABLE, MSG_SYS_DISABLE: enable or disable spontaneous transmission of messages by nodes.
  • MSG_SYS_CLOCK: transmitting the layout time
  • MSG_BOOST_ON, MSG_BOOST_OFF (only if first data byte has a value of 0): setting the booster state
  • MSG_LOCAL_ACCESSORY: Legacy distribution of DCC accessory messages

Note that a newly logged-on node haven't received broadcasts in the past, so this node must be informed individually about system and booster state.

As opposed to host side, the data package will be prepended by a package length-byte at the RS485-side? 
<MAGIC><P_LENGTH><M_LENGTH><MESSAGE>...<M_LENGTH><MESSAGE><CRC><MAGIC>

This is only partially correct. The framing is already guaranteed by the transmission permission through the TOKEN, a MAGIC is therefore no longer required here. The complete package is as follows:

<TOKEN><P_LENGTH><M_LENGTH><MESSAGE>...<M_LENGTH><MESSAGE><CRC>

wherein the token is sent from interface. Even the 'escaping' is omitted at the RS485 side.

What messages will be send from the host to node, after a node has been subscribed at the interface and the interface has been reported this node to the host with MSG_NODE_NEW?
For example, here are the following requests: MSG_SYS_PING, MSG_SYS_GET_SW_VERSION, MSG_SYS_GET_P_VERSION MSG_NODETAB_GETALL, followed by MSG_NODETAB_GETNEXT, which will be sent to the NODETAB after the first answers. MSG_NODETAB_GETNEXT is repeated until the NODETAB is completely read.
For example, the host addresses the node directly with address 0x01 0x00. The interface can forward those requests to the node and release the node with POLL 0x01 for answers. How the response data package will be addressed from the node to the host?
The interface releases the transmission (POLL through a TOKEN where the 9th bit is active) for the desired node. If an answer comes in, the interface knows of course the origin. Then, the interface adds the current node address in front of the message address.
Does the interface also have an address and is the unique_ID of the interface (as a node) also reported to the host?

Yes. The interface has the local address 0 and it will be reported, as well as any other node. The interface itself may also provide features like detector bits or DCC-output.

Also, each node without interface function responds with a table with one entry with the address 0 if he is queried for the node table.

I can't find the address delimiter '0' in the NODETAB.

The NODETAB of an interface reports only the assignment of session address in the bus level of this interface, not the whole address stack. Therefore the address is only one byte large.

A hub has a limited view of the bus, it only knows there is an interface upstream and there are some nodes downstream. All messages from the nodes to the interfaces are forwarded, an the address stack is extended by the current session address of the hub.

How do hubs and those subaddresses work?

A message contains an address stack, this determines how the messages is to be distributed on the network. Example:

4.3.0+message

This is interpreted as follows: The targeted node looks at the first byte of the address, sees a 4 and decides: this is not for me myself, but for sub node 4. So it forwards the packet to sub node 4, but removes the 4 from the stack.

3.0+message

The sub node 4 then looks at the address, sees a 3 and decides: this is not for me myself, but for sub node 3. So it forwards the packet to sub node 3, but removes the 3 from the stack.

0+message

The sub node 3 then looks at the address, sees a 0 – and handles the message.

How is the Unique-ID composed, how can it be evaluated?

The unique_ID consists manufacturer identifier (VID, 8 bits) and a 4-byte (32-bit) number, which contains product identifier (PID) and serial number. It is up to the manufacturer how these 32 bits are split. However, the feature FEATURE_RELEVANT_PID_BITS tells the host system how many bit are relevant for PID.

BiDiB uses little endian, the LSBs are the PID, the upper bits are the serial number.

Example:

Byte 3…7 (containing 32 bits) shall be 0xAA 0xBB 0xCC 0xDD, FEATURE_RELEVANT_PID_BITS shall have a value of 10.

PID: 0x3AA (first byte and two lower bits from byte 2)

Serial: 0x37732E (remaining bits, shifted down by 10)

Code example:


unsigned long pid_and_serial;
unsigned long pid, serial;
pid_and_serial =  le32_to_cpu((unsigned long *)&unique[3]);
pid = pid_and_serial & ( (1 << FEATURE_RELEVANT_PID_BITS) -1 );
serial = pid_and_serial >> FEATURE_RELEVANT_PID_BITS;

le32_to_cpu() is a macro to convert little endian to the internal cpu format.

How is the allocation of these numbers in the vendor ID 13 (DIY according to NMRA) regulated?
For DIY projects, this 32 bit number is divided into two 16-bit numbers. The first two bytes product type identifier, the last two bytes contains a serial number. Each number is transmitted as little endian. The product type identifier will be assigned in coordination with www.opendcc.de.
Are errors always additional messages or they are replacing the normal response?

This is depending on the error message. For example: A node complains about a wrong sequence, but the message is otherwise treated as normal. Now, the node is using this 'wrong' sequence number as a starting point for further sequencing.

Only a error message will be returned if the CRC is wrong and the packet will be discarded. There is no normal response any more.

I always get BIDIB_ERR_SEQUENCE. Why?
a) Each node has its own sequence verification, i.e. the host must maintain the sequence number for each node separately.
b) Sequence is a byte value and simply wraps around, but the value 0 is skipped: ... 253, 254, 255, 1, 2, 3, ...
What will be programmed with the IDENTIFY-button?
The identify button does not program anything, it is used to locate nodes. Since the user doesn't need to assign addresses any more, there must be a possibility to check the assignment between the object at the host software and the BIDIB device: This is done by IDENTIFY: By pressing the IDENTIFY button at the BiDiB device, the corresponding object at the host software should flash. And vice versa: Executing IDENTIFY at the host software results in a LED flashing the BiDiB device.
Is each message transmitted separately?

No. In BiDiB it is intended that messages are bundled. This has the following reasons:

  • The percentage of the protocol overhead increases, the smaller the messages are. Addressing, data safety and so on is constant, the databytes (payload) are little at small messages sizes.
  • The reaction time is better, the smaller the messages.

BiDiB solves this conflict by the fact that messages may be bundled – this means that the protocol increases the efficiency automatically if more data will be transmitted. This data bundling can take place on each transport layer and it can also contain different messages (even from different end-nodes). Bundling will be made only up to the maximum allowed package size of the respective transport layer (BiDiBus e.g. 64 Bytes).

Is it always neccessary to wait for an message answer?

No, you can (and should) combine multiple messages to a node and send them together. However, in deference to the smooth processing of multiple messages in the node these combined messages must not cause more answers than the transmission buffer can ingest. Though that encompasses at least 64 bytes (corresponding to the allowed packet size), the space cannot be completely utilised without troubling spontaneous messages. Therefore, a limit to 48 answer bytes per node is in force.

Example: To query all features of a node you can send MSG_FEATURE_GETALL and 4 MSG_FEATURE_GETNEXT right after the first request. The node starts with the response of MSG_FEATURE_COUNT and sends these 4 features. Immediately after the receipt of MSG_FEATURE_COUNT, another MSG_FEATURE_GETNEXT can be transmitted to keep the connection 'under steam'.

How to prevent a data overrun?

Due to the broadcast nature of BiDiBus, a pc program must pay attention not to override a node with messages. If a node is flooded with messages, it may happen that its answers are not fetched in time by the interface and therefore the node is forced to drop messages or answers. You will notice this by a sequence error.

  • The pc program is allowed to bundle messages to a node up to expected answer size of 48 bytes. To ensure processing, the program shall only do bundling up to an answer size of 43 and shall add a message MSG_PING.
  • Only after receiving the answer of this ping message (or a timeout), the node is addressed again.

A node shall have an output buffer at least twice the size of packet size. Interfaces shall put priority on upstream processing.

In the file BiDiB-Messages.xml the field Answersize denoted the expected size.

How is the displacement of an accessory reported?

An accessory object signals a displacement to the host through a MSG_ACCESSORY_NOTIFY message with an error report, which will also contain the current position as the aspect.

Depending on the internal construction of the node, it might not always be possible to determine the current aspect. In such a case, the node will report the aspect 255. This is more a warning/error that a switch has been manually thrown without the exact condition being known.

After such a report the accessory object always is in an error state and cannot be used. The message shall be acknowledged by the host with a MSG_ACCESSORY_SET that either accepts the new aspect or re-requests the old one. Only after a positive answer (in MSG_ACCESSORY_STATE) the object will be usable again.

Apart from an undesired position discrepancy that is detected with a feedback mechanism, a displacement can also happen through manual on-site operation or through intervention right at the hardware.

Why the change from MSG_LC_KEY to MSG_LC_PORT_STAT, and what do I need to take into consideration?

Revision 1.26 had introduced a generic port model that allows addressing of up to 4096 ports. The input ports have been integrated into this model, thereby allowing to query more than 128 inputs per node and also supporting flexible, convertible port kinds.

Query: Until V0.6, a direct input was prompted with MSG_LC_KEY_QUERY, PORTNUM. From V0.7 this is replaced by MSG_LC_PORT_QUERY PORTADDRESS(16bit). During a period of transition nodes will also support the old query as far as the address range allows it.

Notification: Until V0.6, a direct input was reported with the message MSG_LC_KEY, PORTNUM, STAT. From V0.7 this is replaced by MSG_LC_STAT PORTADDRESS(16bit) STAT. From the view of a PC program, a host can simply accept both message types in parallel.

Rapid query: With the introduction of the MSG_LC_PORT_QUERY_ALL SELECT START END command it is possible to query all port states at system startup. Example: MSG_LC_PORT_QUERY_ALL 00 80 00 00 FF FF queries all inputs (SELECT = 0x8000 = Inputs, START = 0, END = 0xFFFF).