|
The store and forward system provides a reliable way for Ignition to store historical data to the database. Systems such as SQLTags Historian, Transaction Groups, etc. use the store and forward system in order to ensure that data reaches its destination in the database, and is stored in an efficient manner. The store and forward system can be configured in a number of ways, offering both memory buffering for performance and local disk caching for safe storage.
Primary Features and Benefits
The store and forward system offers a number of benefits over other systems that log directly to the database:
| • | Data loss prevention - Data is only removed from the system when the write to the database has executed successfully. |
| • | Guaranteed Ordering - The store and forward system ensures that data is forwarded in the order that it arrived. |
| • | Enhanced performance - By first buffering the data in memory, the store and forward system can optimize writes, and prevent the originating systems from blocking. This means that the system is less likely to lose data samples in the event of system slow downs. |
Store and Forward Data Flow
Although the system offers settings that can affect the pipeline, by default the data flow occurs as follows:
| 1. | Data is generated in some system |
| 2. | Data is placed in a memory buffer |
| 3. | If not removed from memory buffer in some time (the Write Time), or if a certain amount of data accumulates (Write Size), is placed in the local cache. |
| 4. | The data sink, based on a database connection, pulls data in order from the local store, and then the memory buffer, based on the "Write Time" and "Write Size" settings under "Forward Settings". |
| 5. | If the data fails to forward, either due to an error in the connection or in the data itself, it is returned to the buffer or cache. |
| 6. | If the data errors out too many times, it becomes quarantined. |
| 7. | Quarantined data can be managed through the gateway, and can be deleted or un-quarantined, if the error has been resolved. |
Data is forwarded from one stage to the next based on the "Write Time" and "Write Size" triggers. These settings work as an "either/or" manner, meaning that if either of them is surpassed, the data will be forwarded. One important point to note is that the Write Size setting influences the transaction size of similar data to be forwarded, and therefore can have a big impact on performance. As a result, the Write Time should normally be used as the controlling factor, with the Write Size set to something that will provide reasonable transactions, like 100.
|