Electronics-Related.com
Forums

really slow PLL

Started by John Larkin July 20, 2022
On Thursday, 21 July 2022 at 07:49:43 UTC+1, whit3rd wrote:
> On Wednesday, July 20, 2022 at 4:21:08 PM UTC-7, John Larkin wrote: > > Suppose I have several rackmount boxes and each has a BNC connector on > > the back. Each of them has an open-drain mosfet, a weak pullup, and a > > lowpass filtered schmitt gate back into our FPGA. > > > > I can daisy-chain several boxes with BNC cables and tees. > > > > Each box has a 40 MHz VCXO and I want to phase-lock them, or at least > > time-align them to always be the same within a few microseconds, > > longterm. > If you can tolerate 'a few microseconds' on a 40 MHz signal, that's not a phase-lock > problem, it's a frequency-lock problem. Why not just run an up/down counter > to generate a correction voltage for each non-leading VCO?
If you have an ethernet interface to each unit then Precision Time Protocol should do exactly what you want. https://en.wikipedia.org/wiki/Precision_Time_Protocol John
On 21/07/2022 01:22, John Larkin wrote:
> On Wed, 20 Jul 2022 19:32:20 -0400, Phil Hobbs > <pcdhSpamMeSenseless@electrooptical.net> wrote: > >> John Larkin wrote: >>> >>> >>> Suppose I have several rackmount boxes and each has a BNC connector on >>> the back. Each of them has an open-drain mosfet, a weak pullup, and a >>> lowpass filtered schmitt gate back into our FPGA. >>> >>> I can daisy-chain several boxes with BNC cables and tees. >>> >>> Each box has a 40 MHz VCXO and I want to phase-lock them, or at least >>> time-align them to always be the same within a few microseconds, >>> longterm. >>> >>> I could call one the leader (not "master") and make the others >>> followers (not "slaves") and have the leader make an active low pulse >>> maybe once a second. A follower would use her (not "his") clock to >>> measure the incoming period and tweak its local VCXO in the right >>> direction. That should work. >>> >>> Don't GPS receivers lock their 10 MHz oscillators to a 1 PPS pulse >>> from the satellites? >>> >>> My system should work from a 1 PPS GPS pulse too, all boxes as >>> followers. >>> >>> The PLL algorithm might be interesting. >>> >> >> It's certainly possible. However, within whatever tiny loop bandwidth >> you wound up with, the lockers would still have >> >> 20 log(40e6) = 152 dB >> >> higher phase noise than the lockee. > > GPS has that problem too. > >> >> It would be interesting to do the math to see whether it's possible to >> generate a concensus lock for the group: if you get everybody close >> enough, just sum their sine wave outputs and lock each one of them to >> that, with some bit of AC coupling or something so that they don't all >> wander together off to the edge of the tuning range. >> >> Maybe have one doing the locking with a phase shifter and the others >> with VCOs, or something like that. >> >> Definitely a partly-baked idea, but surely one could do better than 152 dB! >> >> Cheers >> >> Phil Hobbs > > Each box is basically a multichannel power supply, but channels can be > programmed to do stuff in timed sequences. I want different box > outputs to time align within, say, one millisecond longterm once > programs are kicked off together. So, many microseconds of equivalent > RMS phase noise is OK as long as we stay time aligned longterm.
You really need to define longterm before the problem becomes well posed. Do you mean hours, days, weeks or months of runtime?
> If a follower is told to start locking, it could timestamp the first > incoming 1 PPS with a giant counter clocked by its local 40 MHz VCO. > If a later 1 PPS edge appears to arrive too soon, we could speed up > our VCXO by, say, 1 PPM, and vice versa. So longterm it walks into > alignment with the 1 PPS and eventually dithers a microsecond per > second. Noise on the coax gets fixed over time too.
Have a free running counter on each of the followers and use the value of that after 1s, 10s, 100s to determine the correct tweak to apply locally. Tweaks of 1ppm at a time is rather crude you should be able to determine the right amount to tweak it by better than that. (especially over longer timebases)
> That's better than just measuring the 1 Hz period once a second, > tweaking the clock, and then throwing away that measurement. I want a > time lock, not a frequency lock.
Then you probably want to measure the cumulative error over many seconds. Each unit can work out how long it can free run without exceeding tolerance once it has the rough and ready count from the first second. After that you have a good idea of how many seconds you can free run for without having any ambiguities from residual drift. This is an ancient trick from physics which avoids the smartest students from having to laboriously count every pendulum swing when determining g to maximum possible precision in a given time. It used to be (and probably still is a favourite exam practical). Components needed are very cheap and the whole thing is a good test of experimental technique. -- Regards, Martin Brown
On Wed, 20 Jul 2022 23:49:40 -0700 (PDT), whit3rd <whit3rd@gmail.com>
wrote:

>On Wednesday, July 20, 2022 at 4:21:08 PM UTC-7, John Larkin wrote: >> Suppose I have several rackmount boxes and each has a BNC connector on >> the back. Each of them has an open-drain mosfet, a weak pullup, and a >> lowpass filtered schmitt gate back into our FPGA. >> >> I can daisy-chain several boxes with BNC cables and tees. >> >> Each box has a 40 MHz VCXO and I want to phase-lock them, or at least >> time-align them to always be the same within a few microseconds, >> longterm. > >If you can tolerate 'a few microseconds' on a 40 MHz signal, that's not a phase-lock >problem, it's a frequency-lock problem. Why not just run an up/down counter >to generate a correction voltage for each non-leading VCO?
It's actually a time lock problem. If a follower box starts up and sees its first 1 PPS input, it can thereafter declare 1 PPS internal events, based on its local VCO, and then do successive early/late comparisons to the external pulses. And trim its VCXO accordingly.
On Thu, 21 Jul 2022 07:43:18 +0200, Gerhard Hoffmann <dk4xp@arcor.de>
wrote:

>Am 21.07.22 um 01:20 schrieb John Larkin: >> >> >> Suppose I have several rackmount boxes and each has a BNC connector on >> the back. Each of them has an open-drain mosfet, a weak pullup, and a >> lowpass filtered schmitt gate back into our FPGA. >> >> I can daisy-chain several boxes with BNC cables and tees. >> >> Each box has a 40 MHz VCXO and I want to phase-lock them, or at least >> time-align them to always be the same within a few microseconds, >> longterm. > >I have a backburner project of locking 16 MTI-260 oscillators >slooowy to another one, and when they are in sync, combine >them with an array of Wilkinsons. That should have a nice >effect on phase noise by averaging over 16. >The CPLD has enough resources to implement that as a delay >locked loop with 1 pps, but low hanging fruit first. > >> >> I could call one the leader (not "master") and make the others >> followers (not "slaves") and have the leader make an active low pulse >> maybe once a second. A follower would use her (not "his") clock to >> measure the incoming period and tweak its local VCXO in the right >> direction. That should work. >> >> Don't GPS receivers lock their 10 MHz oscillators to a 1 PPS pulse >> from the satellites? > >No. There is no 1PPS pulse from the sat nor the need for exactly 10 MHz. >The sats transmit a pseudo noise sequence that is >aligned to the second of their local clock source. >The GPS receiver knows the polynomial and runs a local copy of >the polynomial. It knows by cross correlation if the local >pseudo noise is the same as that of the sat and therefore knows >the start of the second. Usually that won't be the case. >Then the receiver delays its own polynomial by omitting a >clock to the shift register that generates it and tries again. >Sooner or later it will fit.
Where does the 10 MHz come from?
On 21/07/2022 00:20, John Larkin wrote:
> > > Suppose I have several rackmount boxes and each has a BNC connector on > the back. Each of them has an open-drain mosfet, a weak pullup, and a > lowpass filtered schmitt gate back into our FPGA. > > I can daisy-chain several boxes with BNC cables and tees. > > Each box has a 40 MHz VCXO and I want to phase-lock them, or at least > time-align them to always be the same within a few microseconds, > longterm. > > I could call one the leader (not "master") and make the others > followers (not "slaves") and have the leader make an active low pulse > maybe once a second. A follower would use her (not "his") clock to > measure the incoming period and tweak its local VCXO in the right > direction. That should work. > > Don't GPS receivers lock their 10 MHz oscillators to a 1 PPS pulse > from the satellites? > > My system should work from a 1 PPS GPS pulse too, all boxes as > followers. > > The PLL algorithm might be interesting.
Each box has a 40.0something MHz oscillator. Some simple logic triggers on the 1 second GPS pulse and allows 40 million of these clocks through to the rest of the circuit, then waits for the next pulse, rinse and repeat. Of course, this will only work in circumstances where it's ok to stop for a few cycles every second. -- Cheers Clive
On Thu, 21 Jul 2022 12:06:26 +0100, Martin Brown
<'''newspam'''@nonad.co.uk> wrote:

>On 21/07/2022 01:22, John Larkin wrote: >> On Wed, 20 Jul 2022 19:32:20 -0400, Phil Hobbs >> <pcdhSpamMeSenseless@electrooptical.net> wrote: >> >>> John Larkin wrote: >>>> >>>> >>>> Suppose I have several rackmount boxes and each has a BNC connector on >>>> the back. Each of them has an open-drain mosfet, a weak pullup, and a >>>> lowpass filtered schmitt gate back into our FPGA. >>>> >>>> I can daisy-chain several boxes with BNC cables and tees. >>>> >>>> Each box has a 40 MHz VCXO and I want to phase-lock them, or at least >>>> time-align them to always be the same within a few microseconds, >>>> longterm. >>>> >>>> I could call one the leader (not "master") and make the others >>>> followers (not "slaves") and have the leader make an active low pulse >>>> maybe once a second. A follower would use her (not "his") clock to >>>> measure the incoming period and tweak its local VCXO in the right >>>> direction. That should work. >>>> >>>> Don't GPS receivers lock their 10 MHz oscillators to a 1 PPS pulse >>>> from the satellites? >>>> >>>> My system should work from a 1 PPS GPS pulse too, all boxes as >>>> followers. >>>> >>>> The PLL algorithm might be interesting. >>>> >>> >>> It's certainly possible. However, within whatever tiny loop bandwidth >>> you wound up with, the lockers would still have >>> >>> 20 log(40e6) = 152 dB >>> >>> higher phase noise than the lockee. >> >> GPS has that problem too. >> >>> >>> It would be interesting to do the math to see whether it's possible to >>> generate a concensus lock for the group: if you get everybody close >>> enough, just sum their sine wave outputs and lock each one of them to >>> that, with some bit of AC coupling or something so that they don't all >>> wander together off to the edge of the tuning range. >>> >>> Maybe have one doing the locking with a phase shifter and the others >>> with VCOs, or something like that. >>> >>> Definitely a partly-baked idea, but surely one could do better than 152 dB! >>> >>> Cheers >>> >>> Phil Hobbs >> >> Each box is basically a multichannel power supply, but channels can be >> programmed to do stuff in timed sequences. I want different box >> outputs to time align within, say, one millisecond longterm once >> programs are kicked off together. So, many microseconds of equivalent >> RMS phase noise is OK as long as we stay time aligned longterm. > >You really need to define longterm before the problem becomes well >posed. Do you mean hours, days, weeks or months of runtime? > >> If a follower is told to start locking, it could timestamp the first >> incoming 1 PPS with a giant counter clocked by its local 40 MHz VCO. >> If a later 1 PPS edge appears to arrive too soon, we could speed up >> our VCXO by, say, 1 PPM, and vice versa. So longterm it walks into >> alignment with the 1 PPS and eventually dithers a microsecond per >> second. Noise on the coax gets fixed over time too. > >Have a free running counter on each of the followers and use the value >of that after 1s, 10s, 100s to determine the correct tweak to apply >locally. Tweaks of 1ppm at a time is rather crude you should be able to >determine the right amount to tweak it by better than that. >(especially over longer timebases)
I wouldn't expect my VCXO to be more than 10 PPM off at the start of the lock request. So I can walk it to within 1 PPM, namely 1 microsecond error, in at most 10 seconds using 1 PPM jogs. If the osc were 50 PPM off, it would take 50 seconds to catch up to the external pulse.
> >> That's better than just measuring the 1 Hz period once a second, >> tweaking the clock, and then throwing away that measurement. I want a >> time lock, not a frequency lock. > >Then you probably want to measure the cumulative error over many >seconds. Each unit can work out how long it can free run without >exceeding tolerance once it has the rough and ready count from the first >second. After that you have a good idea of how many seconds you can free >run for without having any ambiguities from residual drift.
Yes, I don't want to measure period once a second. I want to compare time alignment forever after receiving the first 1 pps pulse. It's actually simple: first received pulse, start a mod 40 million counter. Every time it rolls over, do an early/late compare to the 1 PPS input, and jog the 40 MHz VCXO 1 PPM in the right direction. The compare is a dflop, d is the msb of the counter, clock is the 1 PPS input. Occasional metastability is fine; it indicates success. It doesn't even need to be a 40 million counter. Something a fraction of that will do. 10,000 maybe. Maybe the counter can just free-run, never get initialized. Gotta do the math on that after I wake up.
Am 21.07.22 um 13:19 schrieb jlarkin@highlandsniptechnology.com:

> > Where does the 10 MHz come from?
Choise of implementer. One local clock generator is needed. This clock determines short term stabiity and phase noise. My Lucent KS24361 uses 5 MHz MTI-260 double ovens; for redundancy/holdover it has a 2nd unit with another crystal oven without a receiver. The redundancy units were really hard to sell without the receiver; that's why I have 20 of these MTI-260, got a good price. :-) They were new old stock built by HP/Agilent for Lucent as replacement parts. They have never been on a telecom tower in China like most of those one gets on ebay. I have expanded the Lucent to 10 MHZ and with a distribution amplifier: < http://www.hoffmann-hochfrequenz.de/downloads/DoubDist.pdf > cheers, Gerhard
jlarkin@highlandsniptechnology.com wrote:
> On Wed, 20 Jul 2022 20:28:35 -0400, Phil Hobbs > <pcdhSpamMeSenseless@electrooptical.net> wrote: > >> John Larkin wrote: >>> On Wed, 20 Jul 2022 19:32:20 -0400, Phil Hobbs >>> <pcdhSpamMeSenseless@electrooptical.net> wrote: >>> >>>> John Larkin wrote: >>>>> >>>>> >>>>> Suppose I have several rackmount boxes and each has a BNC connector on >>>>> the back. Each of them has an open-drain mosfet, a weak pullup, and a >>>>> lowpass filtered schmitt gate back into our FPGA. >>>>> >>>>> I can daisy-chain several boxes with BNC cables and tees. >>>>> >>>>> Each box has a 40 MHz VCXO and I want to phase-lock them, or at least >>>>> time-align them to always be the same within a few microseconds, >>>>> longterm. >>>>> >>>>> I could call one the leader (not "master") and make the others >>>>> followers (not "slaves") and have the leader make an active low pulse >>>>> maybe once a second. A follower would use her (not "his") clock to >>>>> measure the incoming period and tweak its local VCXO in the right >>>>> direction. That should work. >>>>> >>>>> Don't GPS receivers lock their 10 MHz oscillators to a 1 PPS pulse >>>>> from the satellites? >>>>> >>>>> My system should work from a 1 PPS GPS pulse too, all boxes as >>>>> followers. >>>>> >>>>> The PLL algorithm might be interesting. >>>>> >>>> >>>> It's certainly possible. However, within whatever tiny loop bandwidth >>>> you wound up with, the lockers would still have >>>> >>>> 20 log(40e6) = 152 dB >>>> >>>> higher phase noise than the lockee. >>> >>> GPS has that problem too. >>> >>>> >>>> It would be interesting to do the math to see whether it's possible to >>>> generate a concensus lock for the group: if you get everybody close >>>> enough, just sum their sine wave outputs and lock each one of them to >>>> that, with some bit of AC coupling or something so that they don't all >>>> wander together off to the edge of the tuning range. >>>> >>>> Maybe have one doing the locking with a phase shifter and the others >>>> with VCOs, or something like that. >>>> >>>> Definitely a partly-baked idea, but surely one could do better than 152 dB! >>>> >>>> Cheers >>>> >>>> Phil Hobbs >>> >>> Each box is basically a multichannel power supply, but channels can be >>> programmed to do stuff in timed sequences. I want different box >>> outputs to time align within, say, one millisecond longterm once >>> programs are kicked off together. So, many microseconds of equivalent >>> RMS phase noise is OK as long as we stay time aligned longterm. >>> >>> If a follower is told to start locking, it could timestamp the first >>> incoming 1 PPS with a giant counter clocked by its local 40 MHz VCO. >>> If a later 1 PPS edge appears to arrive too soon, we could speed up >>> our VCXO by, say, 1 PPM, and vice versa. So longterm it walks into >>> alignment with the 1 PPS and eventually dithers a microsecond per >>> second. Noise on the coax gets fixed over time too. >>> >>> That's better than just measuring the 1 Hz period once a second, >>> tweaking the clock, and then throwing away that measurement. I want a >>> time lock, not a frequency lock. >>> >> >> Absolutely. The scary 152 dB number doesn't mean that doing something >> like that is automatically a bad idea. >> >> Being an old RF and ultrastable laser guy, though, it does make my ears >> perk up. ;) >> >> Cheers >> >> Phil Hobbs > > I like thermostats, single-bit-feedback control loops. > > We have a couple of boxes that do fan control based on interior > temperature. Once a second, if it's above the setpoint, ratchet fan > speed up some fixed amount, 1% maybe. If it's cooler than the > setpoint, step fan speed down. There's no acoustic drama and it's > perfectly stable. > > It dithers around the setpoint but nobody notices. > > This is immune to classic control theory so the concept annoys some > people but it works great.
A real old time control guy like Tim Wescott would probably be a fan too--the great virtue of a bang-bang controller is that (as you say) it's highly resistant to variations in the _plant_. Your furnace doesn't go nuts when you have a Christmas party, even though all those people generate a lot of heat, and there's lots of opening and closing of doors and ovens. Cheers Phil Hobbs -- Dr Philip C D Hobbs Principal Consultant ElectroOptical Innovations LLC / Hobbs ElectroOptics Optics, Electro-optics, Photonics, Analog Electronics Briarcliff Manor NY 10510 http://electrooptical.net http://hobbs-eo.com
Martin Brown wrote:
> On 21/07/2022 01:22, John Larkin wrote: >> On Wed, 20 Jul 2022 19:32:20 -0400, Phil Hobbs >> <pcdhSpamMeSenseless@electrooptical.net> wrote: >> >>> John Larkin wrote: >>>> >>>> >>>> Suppose I have several rackmount boxes and each has a BNC connector on >>>> the back. Each of them has an open-drain mosfet, a weak pullup, and a >>>> lowpass filtered schmitt gate back into our FPGA. >>>> >>>> I can daisy-chain several boxes with BNC cables and tees. >>>> >>>> Each box has a 40 MHz VCXO and I want to phase-lock them, or at least >>>> time-align them to always be the same within a few microseconds, >>>> longterm. >>>> >>>> I could call one the leader (not "master") and make the others >>>> followers (not "slaves") and have the leader make an active low pulse >>>> maybe once a second. A follower would use her (not "his") clock to >>>> measure the incoming period and tweak its local VCXO in the right >>>> direction. That should work. >>>> >>>> Don't GPS receivers lock their 10 MHz oscillators to a 1 PPS pulse >>>> from the satellites? >>>> >>>> My system should work from a 1 PPS GPS pulse too, all boxes as >>>> followers. >>>> >>>> The PLL algorithm might be interesting. >>>> >>> >>> It's certainly possible.&nbsp; However, within whatever tiny loop bandwidth >>> you wound up with, the lockers would still have >>> >>> 20 log(40e6) = 152 dB >>> >>> higher phase noise than the lockee. >> >> GPS has that problem too. >> >>> >>> It would be interesting to do the math to see whether it's possible to >>> generate a concensus lock for the group: if you get everybody close >>> enough, just sum their sine wave outputs and lock each one of them to >>> that, with some bit of AC coupling or something so that they don't all >>> wander together off to the edge of the tuning range. >>> >>> Maybe have one doing the locking with a phase shifter and the others >>> with VCOs, or something like that. >>> >>> Definitely a partly-baked idea, but surely one could do better than >>> 152 dB! >>> >>> Cheers >>> >>> Phil Hobbs >> >> Each box is basically a multichannel power supply, but channels can be >> programmed to do stuff in timed sequences. I want different box >> outputs to time align within, say, one millisecond longterm once >> programs are kicked off together. So, many microseconds of equivalent >> RMS phase noise is OK as long as we stay time aligned longterm. > > You really need to define longterm before the problem becomes well > posed. Do you mean hours, days, weeks or months of runtime? > >> If a follower is told to start locking, it could timestamp the first >> incoming 1 PPS with a giant counter clocked by its local 40 MHz VCO. >> If a later 1 PPS edge appears to arrive too soon, we could speed up >> our VCXO by, say, 1 PPM, and vice versa. So longterm it walks into >> alignment with the 1 PPS and eventually dithers a microsecond per >> second. Noise on the coax gets fixed over time too. > > Have a free running counter on each of the followers and use the value > of that after 1s, 10s, 100s to determine the correct tweak to apply > locally. Tweaks of 1ppm at a time is rather crude you should be able to > determine the right amount to tweak it by better than that. > (especially over longer timebases) > >> That's better than just measuring the 1 Hz period once a second, >> tweaking the clock, and then throwing away that measurement. I want a >> time lock, not a frequency lock. > > Then you probably want to measure the cumulative error over many > seconds. Each unit can work out how long it can free run without > exceeding tolerance once it has the rough and ready count from the first > second. After that you have a good idea of how many seconds you can free > run for without having any ambiguities from residual drift. > > This is an ancient trick from physics which avoids the smartest students > from having to laboriously count every pendulum swing when determining g > to maximum possible precision in a given time. It used to be (and > probably still is a favourite exam practical). Components needed are > very cheap and the whole thing is a good test of experimental technique. >
It's not as efficient as 'dry labbing'. ;) The tradeoffs are sort of different when you have a $3 CPLD watching things rather than a student. Cheers Phil Hobbs (who didn't work that cheap even back when he was a student) -- Dr Philip C D Hobbs Principal Consultant ElectroOptical Innovations LLC / Hobbs ElectroOptics Optics, Electro-optics, Photonics, Analog Electronics Briarcliff Manor NY 10510 http://electrooptical.net http://hobbs-eo.com
On 7/21/2022 7:06 AM, Martin Brown wrote:
> On 21/07/2022 01:22, John Larkin wrote: >> On Wed, 20 Jul 2022 19:32:20 -0400, Phil Hobbs >> <pcdhSpamMeSenseless@electrooptical.net> wrote: >> >>> John Larkin wrote: >>>> >>>> >>>> Suppose I have several rackmount boxes and each has a BNC connector on >>>> the back. Each of them has an open-drain mosfet, a weak pullup, and a >>>> lowpass filtered schmitt gate back into our FPGA. >>>> >>>> I can daisy-chain several boxes with BNC cables and tees. >>>> >>>> Each box has a 40 MHz VCXO and I want to phase-lock them, or at least >>>> time-align them to always be the same within a few microseconds, >>>> longterm. >>>> >>>> I could call one the leader (not "master") and make the others >>>> followers (not "slaves") and have the leader make an active low pulse >>>> maybe once a second. A follower would use her (not "his") clock to >>>> measure the incoming period and tweak its local VCXO in the right >>>> direction. That should work. >>>> >>>> Don't GPS receivers lock their 10 MHz oscillators to a 1 PPS pulse >>>> from the satellites? >>>> >>>> My system should work from a 1 PPS GPS pulse too, all boxes as >>>> followers. >>>> >>>> The PLL algorithm might be interesting. >>>> >>> >>> It's certainly possible.&nbsp; However, within whatever tiny loop bandwidth >>> you wound up with, the lockers would still have >>> >>> 20 log(40e6) = 152 dB >>> >>> higher phase noise than the lockee. >> >> GPS has that problem too. >> >>> >>> It would be interesting to do the math to see whether it's possible to >>> generate a concensus lock for the group: if you get everybody close >>> enough, just sum their sine wave outputs and lock each one of them to >>> that, with some bit of AC coupling or something so that they don't all >>> wander together off to the edge of the tuning range. >>> >>> Maybe have one doing the locking with a phase shifter and the others >>> with VCOs, or something like that. >>> >>> Definitely a partly-baked idea, but surely one could do better than >>> 152 dB! >>> >>> Cheers >>> >>> Phil Hobbs >> >> Each box is basically a multichannel power supply, but channels can be >> programmed to do stuff in timed sequences. I want different box >> outputs to time align within, say, one millisecond longterm once >> programs are kicked off together. So, many microseconds of equivalent >> RMS phase noise is OK as long as we stay time aligned longterm. > > You really need to define longterm before the problem becomes well > posed. Do you mean hours, days, weeks or months of runtime?
Yeah I don't quite get it, either. My rack of synthesizers can each play one voice of the Maple Leaf Rag via MIDI and they all stay synced together really well, at least over a timespan of several minutes...superficially at least it sounds like he wants a sequencer. Using the nuts & bolts system clock for synchronization of "user tasks" also makes me uncomfortable, if the device behavior only need to align to the millisecond why not trigger them using some simple network protocol and let their hardware figure out how long a millisecond is independently. Do the timings of these boxes need to be tighter than the Maple Leaf Rag?