I'm using a STM32 Cortex M0+ to read an AC signal from a CT. I'm sampling at 6kHz and storing 400 samples. The signal has a DC bias equal to Vcc/2 = 1.65V. In the digital domain this is 2048. In hardware this DC value is very precise, but when sampling it, it varies from 2044 to 2052 inside the buffer. Now if I want to do RMS in that set of data, I need to find a way to deal with this DC bias variation. I have been thinking the following: 1- Subtract a fixed value of 2048 from each ADC reading. This is no so good as I said above this value may vary slightly. Also, if I want to read zero cross it may cause errors to choose exactly 2048 as reference. 2- Sample the DC bias and average it. 3- Don't remove the DC bias. If I calculate RMS then I would get the DC value when there's no input signal. 4- Use a more sophisticated software high pass filter? Example of DC bias readings: ADC_buffer [0] 2048 [1] 2046 [2] 2049 [3] 2051 [4] 2051 [5] 2052 [6] 2050 [7] 2050 [8] 2050 [9] 2047 [10] 2049 [11] 2050 [12] 2049 I appreciate any help or suggestion.
Removing DC offset from ADC Buffer
Started by ●September 20, 2020
Reply by ●September 20, 20202020-09-20
On Sat, 19 Sep 2020 20:38:03 -0700 (PDT), Gold_Spark <bluelectronx@gmail.com> wrote:>I'm using a STM32 Cortex M0+ to read an AC signal from a CT. I'm sampling at 6kHz and storing 400 samples. The signal has a DC bias equal to Vcc/2 = 1.65V. In the digital domain this is 2048. In hardware this DC value is very precise, but when sampling it, it varies from 2044 to 2052 inside the buffer. Now if I want to do RMS in that set of data, I need to find a way to deal with this DC bias variation. > >I have been thinking the following: > >1- Subtract a fixed value of 2048 from each ADC reading. This is no so good as I said above this value may vary slightly. Also, if I want to read zero cross it may cause errors to choose exactly 2048 as reference. >2- Sample the DC bias and average it. >3- Don't remove the DC bias. If I calculate RMS then I would get the DC value when there's no input signal. >4- Use a more sophisticated software high pass filter? > >Example of DC bias readings: > >ADC_buffer >[0] 2048 >[1] 2046 >[2] 2049 >[3] 2051 >[4] 2051 >[5] 2052 >[6] 2050 >[7] 2050 >[8] 2050 >[9] 2047 >[10] 2049 >[11] 2050 >[12] 2049 > >I appreciate any help or suggestion.Just average the samples and subtract that average from each new sample. There are several ways to do that average: Sum the last N samples and divide by N. Exponential smoothing: Avg = Avg + (new-Avg) / N A real higher-order lowpass filter, in software. You could average each 400 points and use that as the "dc bias" to subtract out before doing the RMS, but 400 isn't a lot of points, so the zero value will be noisy. Unless you really have a use for 400 points, there's no need to store any of the samples to get a running RMS. I'd go for the exponential smoothing. Then square each zero-corrected sample, lowpass filter, square root when needed. -- John Larkin Highland Technology, Inc Science teaches us to doubt. Claude Bernard
Reply by ●September 20, 20202020-09-20
On Saturday, September 19, 2020 at 9:49:47 PM UTC-7, jla...@highlandsniptechnology.com wrote:> On Sat, 19 Sep 2020 20:38:03 -0700 (PDT), Gold_Spark > <bluele...@gmail.com> wrote: > > >I'm using a STM32 Cortex M0+ to read an AC signal from a CT. I'm sampling at 6kHz and storing 400 samples. The signal has a DC bias equal to Vcc/2 = 1.65V. In the digital domain this is 2048. In hardware this DC value is very precise, but when sampling it, it varies from 2044 to 2052 inside the buffer. Now if I want to do RMS in that set of data, I need to find a way to deal with this DC bias variation. > > > >I have been thinking the following: > > > >1- Subtract a fixed value of 2048 from each ADC reading. This is no so good as I said above this value may vary slightly. Also, if I want to read zero cross it may cause errors to choose exactly 2048 as reference. > >2- Sample the DC bias and average it. > >3- Don't remove the DC bias. If I calculate RMS then I would get the DC value when there's no input signal. > >4- Use a more sophisticated software high pass filter? > > > >Example of DC bias readings: > > > >ADC_buffer > >[0] 2048 > >[1] 2046 > >[2] 2049 > >[3] 2051 > >[4] 2051 > >[5] 2052 > >[6] 2050 > >[7] 2050 > >[8] 2050 > >[9] 2047 > >[10] 2049 > >[11] 2050 > >[12] 2049 > > > >I appreciate any help or suggestion. > Just average the samples and subtract that average from each new > sample. There are several ways to do that average: > > Sum the last N samples and divide by N.that's a FIR filter (finite impulse response) ... if you choose the sample size and know the likely interference sources (like, 60 Hz ripple), it allows you to place a null appropriately> Exponential smoothing: Avg = Avg + (new-Avg) / Nthat's a IIR filter (infinite impulse response); usually not a great choice> A real higher-order lowpass filter, in software.Like, FIR with weights.
Reply by ●September 20, 20202020-09-20
On Sun, 20 Sep 2020 00:26:47 -0700 (PDT), whit3rd <whit3rd@gmail.com> wrote:>On Saturday, September 19, 2020 at 9:49:47 PM UTC-7, jla...@highlandsniptechnology.com wrote: >> On Sat, 19 Sep 2020 20:38:03 -0700 (PDT), Gold_Spark >> <bluele...@gmail.com> wrote: >> >> >I'm using a STM32 Cortex M0+ to read an AC signal from a CT. I'm sampling at 6kHz and storing 400 samples. The signal has a DC bias equal to Vcc/2 = 1.65V. In the digital domain this is 2048. In hardware this DC value is very precise, but when sampling it, it varies from 2044 to 2052 inside the buffer. Now if I want to do RMS in that set of data, I need to find a way to deal with this DC bias variation. >> > >> >I have been thinking the following: >> > >> >1- Subtract a fixed value of 2048 from each ADC reading. This is no so good as I said above this value may vary slightly. Also, if I want to read zero cross it may cause errors to choose exactly 2048 as reference. >> >2- Sample the DC bias and average it. >> >3- Don't remove the DC bias. If I calculate RMS then I would get the DC value when there's no input signal. >> >4- Use a more sophisticated software high pass filter? >> > >> >Example of DC bias readings: >> > >> >ADC_buffer >> >[0] 2048 >> >[1] 2046 >> >[2] 2049 >> >[3] 2051 >> >[4] 2051 >> >[5] 2052 >> >[6] 2050 >> >[7] 2050 >> >[8] 2050 >> >[9] 2047 >> >[10] 2049 >> >[11] 2050 >> >[12] 2049 >> > >> >I appreciate any help or suggestion. >> Just average the samples and subtract that average from each new >> sample. There are several ways to do that average: >> >> Sum the last N samples and divide by N. > >that's a FIR filter (finite impulse response) >... if you choose the sample size and know the likely interference sources >(like, 60 Hz ripple), it allows you to place a null appropriately > >> Exponential smoothing: Avg = Avg + (new-Avg) / N > >that's a IIR filter (infinite impulse response); usually not a great choiceWhy not? I see a lot of irrational prejuduce against simple IIR filters, in code and in FPGAs. Some people would rather write a hundred lines of code instead of one. If integer math is all that's available, the divide-by-N is just a right shift.> >> A real higher-order lowpass filter, in software. >Like, FIR with weights.Or a multipole IIR. It's fun to do those with just right-shifts too. Again, just a few lines of code. -- John Larkin Highland Technology, Inc Science teaches us to doubt. Claude Bernard
Reply by ●September 20, 20202020-09-20
Gold_Spark <bluelectronx@gmail.com> wrote:> I'm using a STM32 Cortex M0+ to read an AC signal from a CT. I'm sampling at 6kHz and storing 400 samples. The signal has a DC bias equal to Vcc/2 = 1.65V. In the digital domain this is 2048. In hardware this DC value is very precise, but when sampling it, it varies from 2044 to 2052 inside the buffer. Now if I want to do RMS in that set of data, I need to find a way to deal with this DC bias variation. > > I have been thinking the following: > > 1- Subtract a fixed value of 2048 from each ADC reading. This is no so good as I said above this value may vary slightly. Also, if I want to read zero cross it may cause errors to choose exactly 2048 as reference. > 2- Sample the DC bias and average it. > 3- Don't remove the DC bias. If I calculate RMS then I would get the DC value when there's no input signal. > 4- Use a more sophisticated software high pass filter? > > Example of DC bias readings: > > ADC_buffer > [0] 2048 > [1] 2046 > [2] 2049 > [3] 2051 > [4] 2051 > [5] 2052 > [6] 2050 > [7] 2050 > [8] 2050 > [9] 2047 > [10] 2049 > [11] 2050 > [12] 2049 > >Run ADC with DMA continous at a high rate and oversample. Things will get much smoother that way, if your sampling is done right. -- Uwe Bonnes bon@elektron.ikp.physik.tu-darmstadt.de Institut fuer Kernphysik Schlossgartenstrasse 9 64289 Darmstadt --------- Tel. 06151 1623569 ------- Fax. 06151 1623305 ---------
Reply by ●September 20, 20202020-09-20
On 20 Sep 2020 11:50:50 GMT, Uwe Bonnes <bon@hertz.ikp.physik.tu-darmstadt.de> wrote:>Gold_Spark <bluelectronx@gmail.com> wrote: >> I'm using a STM32 Cortex M0+ to read an AC signal from a CT. I'm sampling at 6kHz and storing 400 samples. The signal has a DC bias equal to Vcc/2 = 1.65V. In the digital domain this is 2048. In hardware this DC value is very precise, but when sampling it, it varies from 2044 to 2052 inside the buffer. Now if I want to do RMS in that set of data, I need to find a way to deal with this DC bias variation. >> >> I have been thinking the following: >> >> 1- Subtract a fixed value of 2048 from each ADC reading. This is no so good as I said above this value may vary slightly. Also, if I want to read zero cross it may cause errors to choose exactly 2048 as reference. >> 2- Sample the DC bias and average it. >> 3- Don't remove the DC bias. If I calculate RMS then I would get the DC value when there's no input signal. >> 4- Use a more sophisticated software high pass filter? >> >> Example of DC bias readings: >> >> ADC_buffer >> [0] 2048 >> [1] 2046 >> [2] 2049 >> [3] 2051 >> [4] 2051 >> [5] 2052 >> [6] 2050 >> [7] 2050 >> [8] 2050 >> [9] 2047 >> [10] 2049 >> [11] 2050 >> [12] 2049 >> >> >Run ADC with DMA continous at a high rate and oversample. Things will get much smoother that way, >if your sampling is done right.He still needs to take the DC offset out. Given that the signal comes from a CT, and has no inherent DC component, it can be auto-zeroed by just averaging lots of samples. -- John Larkin Highland Technology, Inc Science teaches us to doubt. Claude Bernard
Reply by ●September 20, 20202020-09-20
On Sunday, September 20, 2020 at 3:26:13 AM UTC-7, jla...@highlandsniptechnology.com wrote:> On Sun, 20 Sep 2020 00:26:47 -0700 (PDT), whit3rd <whi...@gmail.com> > wrote: > >On Saturday, September 19, 2020 at 9:49:47 PM UTC-7, jla...@highlandsniptechnology.com wrote:> >> Just average the samples and subtract that average from each new > >> sample. There are several ways to do that average: > >> > >> Sum the last N samples and divide by N. > > > >that's a FIR filter (finite impulse response) > >... if you choose the sample size and know the likely interference sources > >(like, 60 Hz ripple), it allows you to place a null appropriately > > > >> Exponential smoothing: Avg = Avg + (new-Avg) / N > > > >that's a IIR filter (infinite impulse response); usually not a great choice> Why not? I see a lot of irrational prejuduce against simple IIR > filters, in code and in FPGAs. Some people would rather write a > hundred lines of code instead of one.Oh, it's simple, all right, but it has a long startup transient. That means it doesn't deal with lightning-strike artifacts well, either. In cases (like, reading an analog tape) where you want to reject a particular frequency (the recording bias generator), it doesn't have enough flexibility. FIR was eventually the big winner in CD playback, for instance.
Reply by ●September 20, 20202020-09-20
On Sun, 20 Sep 2020 13:33:11 -0700 (PDT), whit3rd <whit3rd@gmail.com> wrote:>On Sunday, September 20, 2020 at 3:26:13 AM UTC-7, jla...@highlandsniptechnology.com wrote: >> On Sun, 20 Sep 2020 00:26:47 -0700 (PDT), whit3rd <whi...@gmail.com> >> wrote: >> >On Saturday, September 19, 2020 at 9:49:47 PM UTC-7, jla...@highlandsniptechnology.com wrote: > >> >> Just average the samples and subtract that average from each new >> >> sample. There are several ways to do that average: >> >> >> >> Sum the last N samples and divide by N. >> > >> >that's a FIR filter (finite impulse response) >> >... if you choose the sample size and know the likely interference sources >> >(like, 60 Hz ripple), it allows you to place a null appropriately >> > >> >> Exponential smoothing: Avg = Avg + (new-Avg) / N >> > >> >that's a IIR filter (infinite impulse response); usually not a great choice > >> Why not? I see a lot of irrational prejuduce against simple IIR >> filters, in code and in FPGAs. Some people would rather write a >> hundred lines of code instead of one. > >Oh, it's simple, all right, but it has a long startup transient.Any lowpass filter or averager does. Just poke a starting value into the integrator node if you're in a hurry, ADC midscale in this case. Or go for 2nd order. An auto-zero system works fine with a droopy 1st order filter. You have to wait for an FIR filter to fill up, and the output is nonsense until it does. You can poke a start value into it, but you have to load all the nodes.>That means it doesn't deal with lightning-strike artifacts well, either.Presumably an ADC rails on a huge transient. Why would an IIR filter be worse than a FIR for a spike? If you hate long tails, go 2nd order.>In cases (like, reading an analog tape) where you want to reject a >particular frequency (the recording bias generator), it doesn't have enough >flexibility. FIR was eventually the big winner in CD playback, for instance.It's impressive how many convoluted arguments people make to avoid IIR digital filters. Most of them reduce to "It's too simple and I don't like it." -- John Larkin Highland Technology, Inc Science teaches us to doubt. Claude Bernard
Reply by ●September 20, 20202020-09-20
On Sunday, September 20, 2020 at 2:01:05 PM UTC-7, jla...@highlandsniptechnology.com wrote:> On Sun, 20 Sep 2020 13:33:11 -0700 (PDT), whit3rd <whi...@gmail.com> > wrote: > > >On Sunday, September 20, 2020 at 3:26:13 AM UTC-7, jla...@highlandsniptechnology.com wrote: > >> On Sun, 20 Sep 2020 00:26:47 -0700 (PDT), whit3rd <whi...@gmail.com> > >> wrote: > >> >On Saturday, September 19, 2020 at 9:49:47 PM UTC-7, jla...@highlandsniptechnology.com wrote: > > > >> >> Just average the samples and subtract that average from each new > >> >> sample. There are several ways to do that average: > >> >> > >> >> Sum the last N samples and divide by N. > >> > > >> >that's a FIR filter (finite impulse response) > >> >... if you choose the sample size and know the likely interference sources > >> >(like, 60 Hz ripple), it allows you to place a null appropriately > >> > > >> >> Exponential smoothing: Avg = Avg + (new-Avg) / N > >> > > >> >that's a IIR filter (infinite impulse response); usually not a great choice > > > >> Why not? I see a lot of irrational prejuduce against simple IIR > >> filters, in code and in FPGAs. Some people would rather write a > >> hundred lines of code instead of one. > > > >Oh, it's simple, all right, but it has a long startup transient. > Any lowpass filter or averager does. Just poke a starting value into > the integrator node if you're in a hurry, ADC midscale in this case.A lowpass needn't be considered appropriate during startup (and brute-force setting a starting value helps). FIR has a time-limit on its history, which is often completely appropriate and useful.> >That means it doesn't deal with lightning-strike artifacts well, either. > Presumably an ADC rails on a huge transient. Why would an IIR filter > be worse than a FIR for a spike?Small signal in big digitizer range, of course. Your 'rails' scenario is a measurement failure, and there's multiple ways to treat such a thing, which FIR does by... ignoring the spike a few samples afterward. IIR doesn't do that, so saturating the digitizer is an alternate solution that you don't seem to dislike.> It's impressive...I'm pleased that my response impresses you.> how many convoluted arguments people make to avoid IIR > digital filters. Most of them reduce to "It's too simple and I don't > like it."But not any that I mentioned; what ARE those other "convoluted arguments"? I'd like to judge their merits for myself...
Reply by ●September 20, 20202020-09-20
On Sun, 20 Sep 2020 16:29:30 -0700 (PDT), whit3rd <whit3rd@gmail.com> wrote:>On Sunday, September 20, 2020 at 2:01:05 PM UTC-7, jla...@highlandsniptechnology.com wrote: >> On Sun, 20 Sep 2020 13:33:11 -0700 (PDT), whit3rd <whi...@gmail.com> >> wrote: >> >> >On Sunday, September 20, 2020 at 3:26:13 AM UTC-7, jla...@highlandsniptechnology.com wrote: >> >> On Sun, 20 Sep 2020 00:26:47 -0700 (PDT), whit3rd <whi...@gmail.com> >> >> wrote: >> >> >On Saturday, September 19, 2020 at 9:49:47 PM UTC-7, jla...@highlandsniptechnology.com wrote: >> > >> >> >> Just average the samples and subtract that average from each new >> >> >> sample. There are several ways to do that average: >> >> >> >> >> >> Sum the last N samples and divide by N. >> >> > >> >> >that's a FIR filter (finite impulse response) >> >> >... if you choose the sample size and know the likely interference sources >> >> >(like, 60 Hz ripple), it allows you to place a null appropriately >> >> > >> >> >> Exponential smoothing: Avg = Avg + (new-Avg) / N >> >> > >> >> >that's a IIR filter (infinite impulse response); usually not a great choice >> > >> >> Why not? I see a lot of irrational prejuduce against simple IIR >> >> filters, in code and in FPGAs. Some people would rather write a >> >> hundred lines of code instead of one. >> > >> >Oh, it's simple, all right, but it has a long startup transient. >> Any lowpass filter or averager does. Just poke a starting value into >> the integrator node if you're in a hurry, ADC midscale in this case. > >A lowpass needn't be considered appropriate during startup (and >brute-force setting a starting value helps). FIR has a time-limit >on its history, which is often completely appropriate and useful. > > >> >That means it doesn't deal with lightning-strike artifacts well, either. >> Presumably an ADC rails on a huge transient. Why would an IIR filter >> be worse than a FIR for a spike? > >Small signal in big digitizer range, of course. Your 'rails' scenario is >a measurement failure, and there's multiple ways to treat such a thing, >which FIR does by... ignoring the spike a few samples afterward. >IIR doesn't do that, so saturating the digitizer is an alternate solution that >you don't seem to dislike. > >> It's impressive... > >I'm pleased that my response impresses you. > >> how many convoluted arguments people make to avoid IIR >> digital filters. Most of them reduce to "It's too simple and I don't >> like it." > >But not any that I mentioned; what ARE those other "convoluted arguments"? >I'd like to judge their merits for myself...I was just thinking how crazy it woud be to use, say, a 5000-tap FIR filter to compute a good autozero average value of the last 5000 samples. Those 5000 multiply-by-1-and-add blocks will need a lot of logic. -- John Larkin Highland Technology, Inc Science teaches us to doubt. Claude Bernard