On 2019-01-20 15:36, gnuarm.deletethisbit@gmail.com wrote:> On Sunday, January 20, 2019 at 6:06:40 PM UTC-5, Joerg wrote: >> On 2019-01-20 14:49, bitrex wrote: >>> On 01/20/2019 05:26 PM, Joerg wrote: >>>> On 2019-01-20 13:54, Sjouke Burry wrote: >>>>> On 20-1-2019 22:51, Sjouke Burry wrote: >>>>>> On 20-1-2019 21:52, Joerg wrote: >>>>>>> Is there a simple Windows or Linux program that can >>>>>>> decode serial data out of raw data from an digital >>>>>>> oscilloscope? >>>>>>> >>>>>>> My old logic analyzer does not have any serial data >>>>>>> decoding and neither does my scope. So I'd need something >>>>>>> that can take in the ADC raw data, I set a threshold to >>>>>>> turn that into H/L digital values and the software would >>>>>>> fish out the data words. Not sure if it even exist. Not >>>>>>> something expensive because I'll only need it once for a >>>>>>> short time, to find if there's a bug visible in the data. >>>>>>> It'll be a challenge because my scope only has 25,000 >>>>>>> points recording length. >>>>>>> >>>>>> For reading/writing the hardware ports, (for me in DOS) >>>>>> you can download my code. The exe can be loaded on both >>>>>> sides of a nul modem connection, and gives both screen >>>>>> access to the oher computer. >>>>>> >>>>>> It uses the hardware buffer in the rs232 chip, no interrupt >>>>>> used. >>>>>> >>>>> Sorry, I forgot the link.> >>>>> http://home.planet.nl/~burry004/COMPORT.ZIP >>>>> >>>> >>>> Thanks, Sjouke, but unfortunately this isn't coming from the >>>> harware port. It is raw data in the form of an oscilloscope >>>> data dump. Basically 25,000 data words of 8-bits each where a >>>> threshold needs to be set to turn this into high-low digital >>>> data, then the frames need to be found according to start/stop >>>> bits and the result output as ASCII. >>>> >>>> Similar to morse code decoding, just that instead of morse code >>>> it's RS232 data. Or in my case RS485 but it's the same thing. >>>> >>> >>> "Tomorrow" is a pretty tight deadline I'd have to charge a lot >>> for but if you can post a set of example data I could whip up a >>> softwares that might be usable for next time for 'bout $0 >>> >> >> Thanks, but I can't post that publicly. One engineer from the UK >> wrote me a PM and wants to take a stab at it in Excel. I sent him a >> recorded CSV file. I don't yet have the data format info, probably >> one start and one stop bit though I won't know for sure until >> tomorrow. >> >> I can do this one using the old ruler and conting method, hoping >> I'll never need to do this again for the rest of my life. >> >> It just puzzles me that there isn't any software out there for such >> a decoding. I can't be the only one with an older DSO. >> >> Probably this can be done in VBA for Excel but it's been more than >> a decade since I tried anything like that and I am not really a >> programmer. > > I am sure it is available somewhere. It's not really a hard coding > problem. If you know the baud rate in terms of the sample rate the > problem is easy in one pass. If you don't know the baud rate that is > easy enough to find with an initial pass then a second pass to decode > the data. If you want to email me a piece large enough to play with > I'm willing to take a stab at it tonight. >I just sent it but I am too late with it I assume. I'll decipher by hand today but this could be an interesting feature for others as well.> A software UART is the easy part and I know there are lots of them > available. It just needs to be coupled to the CSV data input or > whatever you can provide. Oh, yeah, and do you know the serial port > format? 8 bits, no parity, 1 stop? >That I will have to find out today, looks like 1 start and 1 stop, no parity but I have to confirm. Thanks for the hint about SW UART, I did not even know about that. Will search for it. -- Regards, Joerg http://www.analogconsultants.com/
RS232 decode routines from raw scope data?
Started by ●January 20, 2019
Reply by ●January 21, 20192019-01-21
Reply by ●January 21, 20192019-01-21
On 2019-01-21 05:02, Chris Jones wrote:> On 21/01/2019 10:06, Joerg wrote: >> On 2019-01-20 14:49, bitrex wrote: >>> On 01/20/2019 05:26 PM, Joerg wrote: >>>> On 2019-01-20 13:54, Sjouke Burry wrote: >>>>> On 20-1-2019 22:51, Sjouke Burry wrote: >>>>>> On 20-1-2019 21:52, Joerg wrote: >>>>>>> Is there a simple Windows or Linux program that can decode serial >>>>>>> data >>>>>>> out of raw data from an digital oscilloscope? >>>>>>> >>>>>>> My old logic analyzer does not have any serial data decoding and >>>>>>> neither >>>>>>> does my scope. So I'd need something that can take in the ADC raw >>>>>>> data, >>>>>>> I set a threshold to turn that into H/L digital values and the >>>>>>> software >>>>>>> would fish out the data words. Not sure if it even exist. Not >>>>>>> something >>>>>>> expensive because I'll only need it once for a short time, to >>>>>>> find if >>>>>>> there's a bug visible in the data. It'll be a challenge because my >>>>>>> scope >>>>>>> only has 25,000 points recording length. >>>>>>> >>>>>> For reading/writing the hardware ports, (for me in DOS) you >>>>>> can download my code. >>>>>> The exe can be loaded on both sides of a nul modem connection, >>>>>> and gives both screen access to the oher computer. >>>>>> >>>>>> It uses the hardware buffer in the rs232 chip, no interrupt used. >>>>>> >>>>> Sorry, I forgot the link.> http://home.planet.nl/~burry004/COMPORT.ZIP >>>>> >>>> >>>> Thanks, Sjouke, but unfortunately this isn't coming from the harware >>>> port. It is raw data in the form of an oscilloscope data dump. >>>> Basically 25,000 data words of 8-bits each where a threshold needs to >>>> be set to turn this into high-low digital data, then the frames need >>>> to be found according to start/stop bits and the result output as >>>> ASCII. >>>> >>>> Similar to morse code decoding, just that instead of morse code it's >>>> RS232 data. Or in my case RS485 but it's the same thing. >>>> >>> >>> "Tomorrow" is a pretty tight deadline I'd have to charge a lot for but >>> if you can post a set of example data I could whip up a softwares that >>> might be usable for next time for 'bout $0 >>> >> >> Thanks, but I can't post that publicly. One engineer from the UK wrote >> me a PM and wants to take a stab at it in Excel. I sent him a recorded >> CSV file. I don't yet have the data format info, probably one start >> and one stop bit though I won't know for sure until tomorrow. >> >> I can do this one using the old ruler and conting method, hoping I'll >> never need to do this again for the rest of my life. >> >> It just puzzles me that there isn't any software out there for such a >> decoding. I can't be the only one with an older DSO. > Sigrok can do it. https://sigrok.org/ > That software supports cheap USB logic analysers (I know you don't have > one, so no good for this time, but you should spend the $6.86 so that > you have one ready for next time): > https://www.dx.com/p/logic-analyzer-w-dupont-lines-and-usb-cable-for-scm-black-2017429 > > I understand that Sigrok also supports many DSOs, and may support the > one that you have, or some intermediate format that you can save to. > > It is a while since I installed Sigrok (on linux). I seem to recall > needing a fairly recent version of linux and Sigrok, though I didn't > have to build anything from source. It was a pain to set up, but saved > me a lot of time in the end. I think I used a nightly build appimage: > https://sigrok.org/jenkins/job/sigrok-native-appimage/platform=native-i686-appimage/lastSuccessfulBuild/artifact/cross-compile/appimage/out/PulseView-NIGHTLY-i686.AppImage > >Thanks. At first glance it looks like it does RS232 only via a Prolific chip which wouldn't allow my measurements because I have to trigger on an event: https://sigrok.org/wiki/Protocol_decoder:Uart IOW, I can't just hook a RS232-USB converter to a PC and log. Unless I somehow hardwire the scope's trigger output to a pass device that lets the stream start only after a trigger event. -- Regards, Joerg http://www.analogconsultants.com/
Reply by ●January 21, 20192019-01-21
On 2019-01-20 18:14, mpm wrote:> On Sunday, January 20, 2019 at 5:26:16 PM UTC-5, Joerg wrote: > > Joerg, from memory, RS-232 uses inverted logic and I think it is > transmitted LSB first. You'll definitely want to verify that before > you get too far along. >I'll have to verify the data format today.> Also, on your scope output, are the 25k samples synchronized (for > lack of a better work) to the baud rate of the serial message? Or do > you have a situation where you might have dozen of scope readings in > the CSV file corresponding with the single high (or low) transition > of an individual bit of the serial message? >It's non-synchronized but grossly oversampled so that the detection should be easy. Essentially 25,000 8-bit sample points where low is around zero and high is around 80.> If the latter,... Ugh! Still manageable, but adds an extra layer of > complexity to the task. > > FWIW: A cheap RIGOL scope ($350-ish USD) can be had with built-in > RS232 decoding, among other popular format. Not sure how deep the > recording is, though since I've never really used that feature past a > few dozen or so characters. >Unfortunately too late and my Instek GDS-2204 is of a generation that didn't have this yet.> I sort of like the Martin's idea of running the file through a sound > card or maybe even a modem, and recover the data using a dumb > terminal or terminal emulator (i.e., putty). If you ignore all the > error correction, in the end it's just tones. > > I wonder if there's a program out there that will decode RS232 when > presented as a wav or mp3 file? >That is pretty much what I am looking for. Should not be a big deal to turn a CSV file into WAV, I hope. The other idea would be to cram the file into LTSpice and see if there's any way to decode it using the available logic models. Though that would become a project on its own. -- Regards, Joerg http://www.analogconsultants.com/
Reply by ●January 21, 20192019-01-21
On 2019-01-20 20:49, Jasen Betts wrote:> On 2019-01-20, Joerg <news@analogconsultants.com> wrote: >> On 2019-01-20 13:54, Sjouke Burry wrote: >>> On 20-1-2019 22:51, Sjouke Burry wrote: >>>> On 20-1-2019 21:52, Joerg wrote: >>>>> Is there a simple Windows or Linux program that can decode serial data >>>>> out of raw data from an digital oscilloscope? >>>>> >>>>> My old logic analyzer does not have any serial data decoding and neither >>>>> does my scope. So I'd need something that can take in the ADC raw data, >>>>> I set a threshold to turn that into H/L digital values and the software >>>>> would fish out the data words. Not sure if it even exist. Not something >>>>> expensive because I'll only need it once for a short time, to find if >>>>> there's a bug visible in the data. It'll be a challenge because my scope >>>>> only has 25,000 points recording length. >>>>> >>>> For reading/writing the hardware ports, (for me in DOS) you >>>> can download my code. >>>> The exe can be loaded on both sides of a nul modem connection, >>>> and gives both screen access to the oher computer. >>>> >>>> It uses the hardware buffer in the rs232 chip, no interrupt used. >>>> >>> Sorry, I forgot the link.> http://home.planet.nl/~burry004/COMPORT.ZIP >>> >> >> Thanks, Sjouke, but unfortunately this isn't coming from the harware >> port. It is raw data in the form of an oscilloscope data dump. Basically >> 25,000 data words of 8-bits each where a threshold needs to be set to >> turn this into high-low digital data, then the frames need to be found >> according to start/stop bits and the result output as ASCII. > > squirt it out the sound card, amplify it, schmitt it, and pull it > back in through the UART. > > > else you'll need to write a software uart. >Theoretically possible, with some transistor stuff. -- Regards, Joerg http://www.analogconsultants.com/
Reply by ●January 21, 20192019-01-21
On 2019-01-20 16:05, Tim Williams wrote:> https://softwareengineering.stackexchange.com/questions/153351/is-there-an-specific-way-or-algorithm-to-decode-protocols > > ? > > Tons of examples of software decode for embedded platforms, might take > longer to refactor for a data series than to write one new though? > > I'd gladly write one say in JavaScript and put it up on my website for > all to use, but I don't need one, and I'm guessing you don't have the > time/budget to do that on request. > > Well, such is life :) >Yeah, time is the issue and I am now to the point where I pretty much have to decode by hand. -- Regards, Joerg http://www.analogconsultants.com/
Reply by ●January 21, 20192019-01-21
On Mon, 21 Jan 2019 07:17:35 -0800, Joerg <news@analogconsultants.com> wrote:>On 2019-01-20 15:00, John Larkin wrote: >> On Sun, 20 Jan 2019 13:43:12 -0800, Joerg <news@analogconsultants.com> >> wrote: >> >>> On 2019-01-20 13:21, Lasse Langwadt Christensen wrote: >>>> s�ndag den 20. januar 2019 kl. 21.52.53 UTC+1 skrev Joerg: >>>>> Is there a simple Windows or Linux program that can decode serial >>>>> data out of raw data from an digital oscilloscope? >>>>> >>>>> My old logic analyzer does not have any serial data decoding and >>>>> neither does my scope. So I'd need something that can take in the >>>>> ADC raw data, I set a threshold to turn that into H/L digital >>>>> values and the software would fish out the data words. Not sure if >>>>> it even exist. Not something expensive because I'll only need it >>>>> once for a short time, to find if there's a bug visible in the >>>>> data. It'll be a challenge because my scope only has 25,000 points >>>>> recording length. >>>> >>>> get a cheap logic usb analyser? >>>> https://learn.sparkfun.com/tutorials/using-the-usb-logic-analyzer-with-sigrok-pulseview/all >>>> >>> >>> That sure would ne nice but I have to do this tomorrow. >>> >>> >>>> else it would take long to program, just slice that data into 1/0 and >>>> grap/decode a frame worth of data after each start bit >>>> >>> >>> I thought there had to be some programs that already can do that. Many >>> times people sit there and decode by hand. I used to do that in my olden >>> days, using a ruler and stacks of Polaroid pictures from a scope screen. >>> >>> It should even be possible to do this in an Excel macro and then load in >>>from the comma-delimited data file but I am not that great a programmer, >>> I am an analog guy. >>> >>> >>>> what's the format of the data from the scope? >>>> >>> >>> It can store raw CSV for the data. Also BMP, JPEG and some others but >>> those only per screen and not the whole recording. >>> >>> If there is nothing I'll just use the old print-out, ruler and counting >>> method. >> >> Why not connect your RS232 to a serial port on a PC or laptop? >> > >It comes in the form of logic level inside a circuit. I can try to whip >up some sauerkraut electronics to transfer that to bipolar RS232. One of >the wee challenges would be to sync two streams, TX and RX. With a scope >that's easy, I can just trigger on an event on the TX line and record >what RX responds with.Most RS232 receivers work fine with TTL input (unless it's inverted!) -- John Larkin Highland Technology, Inc trk jlarkin att highlandtechnology dott com http://www.highlandtechnology.com
Reply by ●January 21, 20192019-01-21
On 2019-01-20 16:49, Martin Riddle wrote:> On Sun, 20 Jan 2019 12:52:48 -0800, Joerg <news@analogconsultants.com> > wrote: > >> Is there a simple Windows or Linux program that can decode serial data >> out of raw data from an digital oscilloscope? >> >> My old logic analyzer does not have any serial data decoding and neither >> does my scope. So I'd need something that can take in the ADC raw data, >> I set a threshold to turn that into H/L digital values and the software >> would fish out the data words. Not sure if it even exist. Not something >> expensive because I'll only need it once for a short time, to find if >> there's a bug visible in the data. It'll be a challenge because my scope >> only has 25,000 points recording length. > > How abot sending it out the sound card and into a rs232 input and > reading it ito Procomm or some DOS/Win serial program. You'll need to > get the timing right for the baud rate to get it to work. >Could be done but would require some fidgeting with hardware as well. I'll see. -- Regards, Joerg http://www.analogconsultants.com/
Reply by ●January 21, 20192019-01-21
On Monday, January 21, 2019 at 10:44:34 AM UTC-5, Joerg wrote:> On 2019-01-20 16:05, Tim Williams wrote: > > https://softwareengineering.stackexchange.com/questions/153351/is-there-an-specific-way-or-algorithm-to-decode-protocols > > > > ? > > > > Tons of examples of software decode for embedded platforms, might take > > longer to refactor for a data series than to write one new though? > > > > I'd gladly write one say in JavaScript and put it up on my website for > > all to use, but I don't need one, and I'm guessing you don't have the > > time/budget to do that on request. > > > > Well, such is life :) > > > > Yeah, time is the issue and I am now to the point where I pretty much > have to decode by hand.Not really. If you send me a file I will write a program to pull out the data. What format do you want the output? Rick C. + Get 6 months of free supercharging + Tesla referral code - https://ts.la/richard11209
Reply by ●January 21, 20192019-01-21
On Jan 21, 2019, Joerg wrote (in article <gam6bsFduutU3@mid.individual.net>):> On 2019-01-20 16:03, Joseph Gwinn wrote: > > On Jan 20, 2019, Joerg wrote > > (in article <gak8bgF147uU1@mid.individual.net>): > > > > > On 2019-01-20 13:21, Lasse Langwadt Christensen wrote: > > > > s�ndag den 20. januar 2019 kl. 21.52.53 UTC+1 skrev Joerg: > > > > > Is there a simple Windows or Linux program that can decode serial > > > > > data out of raw data from an digital oscilloscope? > > > > > > > > > > My old logic analyzer does not have any serial data decoding and > > > > > neither does my scope. So I'd need something that can take in the > > > > > ADC raw data, I set a threshold to turn that into H/L digital > > > > > values and the software would fish out the data words. Not sure if > > > > > it even exist. Not something expensive because I'll only need it > > > > > once for a short time, to find if there's a bug visible in the > > > > > data. It'll be a challenge because my scope only has 25,000 points > > > > > recording length. > > > > > > > > get a cheap logic usb analyser? > > > > https://learn.sparkfun.com/tutorials/using-the-usb-logic-analyzer-with-sg> > > > ro > > > > k-pulseview/all > > > > > > That sure would ne nice but I have to do this tomorrow. > > > > > > > else it would take long to program, just slice that data into 1/0 and > > > > grap/decode a frame worth of data after each start bit > > > > > > I thought there had to be some programs that already can do that. Many > > > times people sit there and decode by hand. I used to do that in my olden > > > days, using a ruler and stacks of Polaroid pictures from a scope screen. > > > > > > It should even be possible to do this in an Excel macro and then load in > > > from the comma-delimited data file but I am not that great a programmer, > > > I am an analog guy. > > > > > > > what's the format of the data from the scope? > > > > > > It can store raw CSV for the data. Also BMP, JPEG and some others but > > > those only per screen and not the whole recording. > > > > > > If there is nothing I'll just use the old print-out, ruler and counting > > > method. > > > > CSV is a standard database forms. Many MathCad programs will accept CSV, > > such as Mathematica, Matlab, and of course MS Excel. > > > > Excel will plot results, and actually has a FFT function, so Excel may be > > your only choice given the one-day limit. > > I'll probably do it by hand. For a non-programmer like me that's usually > faster than trying to learn VBA and writing code.What are you going to do in one day to process 25,000 data points by hand? I bet a plot will help a lot. I think you are over-thinking this. No VBA programming needed. Just try it, using standard Excel functions. Or, just use Excel a a format converter. Joe Gwinn
Reply by ●January 21, 20192019-01-21
On 2019-01-21 07:55, Joseph Gwinn wrote:> On Jan 21, 2019, Joerg wrote > (in article <gam6bsFduutU3@mid.individual.net>): > >> On 2019-01-20 16:03, Joseph Gwinn wrote: >>> On Jan 20, 2019, Joerg wrote >>> (in article <gak8bgF147uU1@mid.individual.net>): >>> >>>> On 2019-01-20 13:21, Lasse Langwadt Christensen wrote: >>>>> s�ndag den 20. januar 2019 kl. 21.52.53 UTC+1 skrev Joerg: >>>>>> Is there a simple Windows or Linux program that can decode serial >>>>>> data out of raw data from an digital oscilloscope? >>>>>> >>>>>> My old logic analyzer does not have any serial data decoding and >>>>>> neither does my scope. So I'd need something that can take in the >>>>>> ADC raw data, I set a threshold to turn that into H/L digital >>>>>> values and the software would fish out the data words. Not sure if >>>>>> it even exist. Not something expensive because I'll only need it >>>>>> once for a short time, to find if there's a bug visible in the >>>>>> data. It'll be a challenge because my scope only has 25,000 points >>>>>> recording length. >>>>> >>>>> get a cheap logic usb analyser? >>>>> https://learn.sparkfun.com/tutorials/using-the-usb-logic-analyzer-with-s > g >>>>> ro >>>>> k-pulseview/all >>>> >>>> That sure would ne nice but I have to do this tomorrow. >>>> >>>>> else it would take long to program, just slice that data into 1/0 and >>>>> grap/decode a frame worth of data after each start bit >>>> >>>> I thought there had to be some programs that already can do that. Many >>>> times people sit there and decode by hand. I used to do that in my olden >>>> days, using a ruler and stacks of Polaroid pictures from a scope screen. >>>> >>>> It should even be possible to do this in an Excel macro and then load in >>>> from the comma-delimited data file but I am not that great a programmer, >>>> I am an analog guy. >>>> >>>>> what's the format of the data from the scope? >>>> >>>> It can store raw CSV for the data. Also BMP, JPEG and some others but >>>> those only per screen and not the whole recording. >>>> >>>> If there is nothing I'll just use the old print-out, ruler and counting >>>> method. >>> >>> CSV is a standard database forms. Many MathCad programs will accept CSV, >>> such as Mathematica, Matlab, and of course MS Excel. >>> >>> Excel will plot results, and actually has a FFT function, so Excel may be >>> your only choice given the one-day limit. >> >> I'll probably do it by hand. For a non-programmer like me that's usually >> faster than trying to learn VBA and writing code. > > What are you going to do in one day to process 25,000 data points by hand? I > bet a plot will help a lot. >It's grossly oversampled so there are only a few dozen data words in that stream. The way I did that in the past was to make a paper ruler to scale, scroll the trace across the screen and figure out the hex code with a paper look-up table. Of course, now when fast-forwarding 25 years one has to first lay packaging plastic over the screen because they aren't glass anymore and could scratch.> I think you are over-thinking this. No VBA programming needed. Just try it, > using standard Excel functions. Or, just use Excel a a format converter. >I dnt know which standard Excel funtions or format converters would convert a scope raw data output to hex. Martin and Lasse sent me a VBA rountine and C code respectively and the hex data that came out of both methods was the same, which is very encouraging. -- Regards, Joerg http://www.analogconsultants.com/
