DIS currently runs a single process on a single server. If DIS crashes, the server crashes or another issue occurs how can I:
Hi Ian, thanks for your question!
DIS can be installed as a Windows Service to ensure its status is managed by Windows in case the software crashes. The built-in Services app available in all Windows versions (accessible via the run command using services.msc) can be used to monitor the status of the agent, as well as to configure its start-up type (e.g. automatic or manual).
In regards to using multiple DIS instances for better redundancy, we don’t offer a first-party solution built into the software but it’s relatively easy to implement when using the “Advanced mode”, as it allows you to use any Node.js package available via NPM.
As an example, you could be using the redislock package to use Redis as a shared system between multiple instances to acquire locks and only run scheduled operations on up to one node at any time.
Alternatively, you can set up multiple instances to carry out a sync at different times (e.g. one instance at .00 minutes of each hour and the other at .30 minutes) using the frequency property as documented on the DIS documentation and use the Windows Events Viewer to monitor the output of all nodes being online. This is fairly easy to achieve as it doesn’t require a complex setup involving new pieces of infrastructure.
Using Windows Events Viewer you can also set up notifications such as emails when errors are detected. Open any DIS-triggered error event on Windows Events Viewer and click on “Attach a Task to this Log” to set up alerts when such error is triggered. The list of messages logged by the agent can be seen here: Data integration service | Fliplet Developers Documentation
Nick,
Thanks, great suggestions. It sounds like Redislock requires Redis which may not be possible for all customers. Are there any solutions that use SQL Server/sequalize or files?
The 2nd option you suggested with DIS instances running at different times sounds easier plus it can be paired with email alerts from Windows Events Viewer.
I agree having multiple instances carrying out the sync at different times is the recommended option given its simplicity both in setup and maintenance required.
In relation to alternatives for a distributed lock manager, NPM provides plenty of options for a variety of adapters: distributed locking - npm search
As an example, this library uses Sequelize to provide a dialect-agnostic library to easily implement locks.
Let me know if you need more guidance with the setup!