We use the efinix T20 trion FPGA. Questions about the config bit streams: Are they always the same size, or does it depend on how much logic is compiled? Would a simple application use less? Are the streams very compressible? We have done some simple run-length coding to greatly reduce the storage requirement for other FPGAs. Configs tend to have long runs of 0's. The T20/256 claims to need 5.4 megabits. I'd like to store the fpga config and application code in a Raspberry Pi Pico, which has 2 MB of onboard flash. Storing the full config would use about a third of that, so reducing that would be useful.
efinix bit stream question
Started by ●November 27, 2022
Reply by ●November 27, 20222022-11-27
On Sunday, November 27, 2022 at 12:34:31 AM UTC-4, John Larkin wrote:> We use the efinix T20 trion FPGA. > > Questions about the config bit streams: > > Are they always the same size, or does it depend on how much logic is > compiled? Would a simple application use less?I don't know about Efinix, but bitstream compression is not unusual in the industry. Bitstreams tend to have a lot of compressibility without using fancy algorithms. It's easy enough to test. Construct a simple design and compile one instance, look at the size, then instantiate multiple instances and check the size again. There is likely a control somewhere to enable/disable compression if it's available.> Are the streams very compressible? We have done some simple run-length > coding to greatly reduce the storage requirement for other FPGAs. > Configs tend to have long runs of 0's.So you compress for storage and expand prior to downloading? I assume this is done on the fly? RLL encoding?> The T20/256 claims to need 5.4 megabits. I'd like to store the fpga > config and application code in a Raspberry Pi Pico, which has 2 MB of > onboard flash. Storing the full config would use about a third of > that, so reducing that would be useful.2 MB is pretty small these days. The trouble with compression is it may not provide much reduction in size as the design fills up, but then again, maybe it still does. Efinix is the one with little dedicated routing, instead using the logic elements for routing, right? I've not done enough research to tell how large a part is needed for a given size design. I recall a line of FPGAs from Atmel that was like that. It was not so good in the end. I think, before I used any of their parts, I would want to compile a design using tools from another, conventional FPGA maker and Efinix and see how they compare. Gowin is a lot less expensive. So far, I like them. But they are Chinese, so I may not be able to use their parts. -- Rick C. - Get 1,000 miles of free Supercharging - Tesla referral code - https://ts.la/richard11209
Reply by ●November 27, 20222022-11-27
Am 27.11.22 um 05:34 schrieb John Larkin:> We use the efinix T20 trion FPGA. > > Questions about the config bit streams: > > Are they always the same size, or does it depend on how much logic is > compiled? Would a simple application use less?With Xilinx it would for sure. Never used efinix, but I would consider it broken if it didn't.> Are the streams very compressible?I would simply test example files with zip, zcat and similar. IIRC, there is even a flow-through decompressor. We have done some simple run-length> coding to greatly reduce the storage requirement for other FPGAs. > Configs tend to have long runs of 0's. > > The T20/256 claims to need 5.4 megabits. I'd like to store the fpga > config and application code in a Raspberry Pi Pico, which has 2 MB of > onboard flash. Storing the full config would use about a third of > that, so reducing that would be useful.cheers, Gerhard
Reply by ●November 27, 20222022-11-27
søndag den 27. november 2022 kl. 10.46.54 UTC+1 skrev Gerhard Hoffmann:> Am 27.11.22 um 05:34 schrieb John Larkin: > > We use the efinix T20 trion FPGA. > > > > Questions about the config bit streams: > > > > Are they always the same size, or does it depend on how much logic is > > compiled? Would a simple application use less? > With Xilinx it would for sure. Never used efinix, but I would > consider it broken if it didn't.Xilinx only if you use bit file compression and don't use encryption
Reply by ●November 27, 20222022-11-27
On Sun, 27 Nov 2022 10:46:47 +0100, Gerhard Hoffmann <dk4xp@arcor.de> wrote:>Am 27.11.22 um 05:34 schrieb John Larkin: >> We use the efinix T20 trion FPGA. >> >> Questions about the config bit streams: >> >> Are they always the same size, or does it depend on how much logic is >> compiled? Would a simple application use less? > >With Xilinx it would for sure. Never used efinix, but I would >consider it broken if it didn't. > >> Are the streams very compressible? > >I would simply test example files with zip, zcat and similar. >IIRC, there is even a flow-through decompressor. > >We have done some simple run-length >> coding to greatly reduce the storage requirement for other FPGAs. >> Configs tend to have long runs of 0's. >> >> The T20/256 claims to need 5.4 megabits. I'd like to store the fpga >> config and application code in a Raspberry Pi Pico, which has 2 MB of >> onboard flash. Storing the full config would use about a third of >> that, so reducing that would be useful. > >cheers, Gerhard >I'm at home and don't have access to a compiled bitstream, and this is a discussion group. I'll get a T20 bit stream Monday or Tuesday and see what it looks like. If there are many runs of 0's, compression and decompression are very simple. Or maybe a typical stream is just shorter than the max. I recall a Xilinx or maybe Altera stream that compressed about 3:1 with a very simple algorithm. I think I compressed runs of 0's and 1's on that one, with a PowerBasic program. We considered fancier dictionary-based schemes, sort of like Zip, but they weren't worth the hassle.
Reply by ●November 27, 20222022-11-27
On Sun, 27 Nov 2022 08:16:57 -0800, John Larkin <jlarkin@highlandSNIPMEtechnology.com> wrote:>On Sun, 27 Nov 2022 10:46:47 +0100, Gerhard Hoffmann <dk4xp@arcor.de> >wrote: > >>Am 27.11.22 um 05:34 schrieb John Larkin: >>> We use the efinix T20 trion FPGA. >>> >>> Questions about the config bit streams: >>> >>> Are they always the same size, or does it depend on how much logic is >>> compiled? Would a simple application use less? >> >>With Xilinx it would for sure. Never used efinix, but I would >>consider it broken if it didn't. >> >>> Are the streams very compressible? >> >>I would simply test example files with zip, zcat and similar. >>IIRC, there is even a flow-through decompressor. >> >>We have done some simple run-length >>> coding to greatly reduce the storage requirement for other FPGAs. >>> Configs tend to have long runs of 0's. >>> >>> The T20/256 claims to need 5.4 megabits. I'd like to store the fpga >>> config and application code in a Raspberry Pi Pico, which has 2 MB of >>> onboard flash. Storing the full config would use about a third of >>> that, so reducing that would be useful. >> >>cheers, Gerhard >> > >I'm at home and don't have access to a compiled bitstream, and this is >a discussion group. > >I'll get a T20 bit stream Monday or Tuesday and see what it looks >like. If there are many runs of 0's, compression and decompression are >very simple. Or maybe a typical stream is just shorter than the max. > >I recall a Xilinx or maybe Altera stream that compressed about 3:1 >with a very simple algorithm. I think I compressed runs of 0's and 1's >on that one, with a PowerBasic program. > >We considered fancier dictionary-based schemes, sort of like Zip, but >they weren't worth the hassle. > >I recall the conclusion that the best dictionary entry for a random data block is itself. Zip doesn't compress random binary data files very well. FPGA bit streams are nonrandom in having long runs of 0's.
Reply by ●December 1, 20222022-12-01
On Sat, 26 Nov 2022 20:34:23 -0800, John Larkin <jlarkin@highlandSNIPMEtechnology.com> wrote:>We use the efinix T20 trion FPGA. > >Questions about the config bit streams: > >Are they always the same size, or does it depend on how much logic is >compiled? Would a simple application use less? > >Are the streams very compressible? We have done some simple run-length >coding to greatly reduce the storage requirement for other FPGAs. >Configs tend to have long runs of 0's. > >The T20/256 claims to need 5.4 megabits. I'd like to store the fpga >config and application code in a Raspberry Pi Pico, which has 2 MB of >onboard flash. Storing the full config would use about a third of >that, so reducing that would be useful.Here's a T20 bit stream. The length seems to be constant vs functions coded, but there are enough runs of all 0's that it's probably worth compressing. https://www.dropbox.com/s/vm247lntp78jm20/Efinix_T20_bitstream.hex?dl=0 The actual config file will be binary, not hex of course.
Reply by ●December 1, 20222022-12-01
On 2/12/22 08:12, John Larkin wrote:> On Sat, 26 Nov 2022 20:34:23 -0800, John Larkin > <jlarkin@highlandSNIPMEtechnology.com> wrote: > >> We use the efinix T20 trion FPGA. >> >> Questions about the config bit streams: >> >> Are they always the same size, or does it depend on how much logic is >> compiled? Would a simple application use less? >> >> Are the streams very compressible? We have done some simple run-length >> coding to greatly reduce the storage requirement for other FPGAs. >> Configs tend to have long runs of 0's. >> >> The T20/256 claims to need 5.4 megabits. I'd like to store the fpga >> config and application code in a Raspberry Pi Pico, which has 2 MB of >> onboard flash. Storing the full config would use about a third of >> that, so reducing that would be useful. > > Here's a T20 bit stream. The length seems to be constant vs functions > coded, but there are enough runs of all 0's that it's probably worth > compressing. > > https://www.dropbox.com/s/vm247lntp78jm20/Efinix_T20_bitstream.hex?dl=0 > > The actual config file will be binary, not hex of course.Gzip compresses your 2.0MB down to 105kB. The decompressor isn't tiny, but it's fairly small. The lz4 decompressor is tiny and still gets to 221kB. Possibly less if you RLE first. bz2 gets it to 76kB, and xz or lzma to 72kB. Compression is one area where it's best to rely on work done by people who understand the theory. Some of these algorithms have a tiny decompressor, the magic is in the compressor. CH
Reply by ●December 1, 20222022-12-01
On Thursday, December 1, 2022 at 4:51:08 PM UTC-5, Clifford Heath wrote:> On 2/12/22 08:12, John Larkin wrote: > > On Sat, 26 Nov 2022 20:34:23 -0800, John Larkin > > <jla...@highlandSNIPMEtechnology.com> wrote: > > > >> We use the efinix T20 trion FPGA. > >> > >> Questions about the config bit streams: > >> > >> Are they always the same size, or does it depend on how much logic is > >> compiled? Would a simple application use less? > >> > >> Are the streams very compressible? We have done some simple run-length > >> coding to greatly reduce the storage requirement for other FPGAs. > >> Configs tend to have long runs of 0's. > >> > >> The T20/256 claims to need 5.4 megabits. I'd like to store the fpga > >> config and application code in a Raspberry Pi Pico, which has 2 MB of > >> onboard flash. Storing the full config would use about a third of > >> that, so reducing that would be useful. > > > > Here's a T20 bit stream. The length seems to be constant vs functions > > coded, but there are enough runs of all 0's that it's probably worth > > compressing. > > > > https://www.dropbox.com/s/vm247lntp78jm20/Efinix_T20_bitstream.hex?dl=0 > > > > The actual config file will be binary, not hex of course. > Gzip compresses your 2.0MB down to 105kB. The decompressor isn't tiny, > but it's fairly small. The lz4 decompressor is tiny and still gets to > 221kB. Possibly less if you RLE first. bz2 gets it to 76kB, and xz or > lzma to 72kB. > > Compression is one area where it's best to rely on work done by people > who understand the theory. Some of these algorithms have a tiny > decompressor, the magic is in the compressor.Not really. If you need to send the data to Voyager 1, then every bit matters. For this work, there is no need for "best", or even "very good". Accommodation must be made for variety in the design, including growth. There is every reason to think that a design with more used elements would compress less. So if a simple compression method (such as RLL) gets the design down to 400 kB, there's not much need to get it down to 100 kB. Compression works by finding repeatability in the data. In this case, the best compression would likely be an algorithm that is aware of the structure of the bit stream. In other words, a custom. Since "best" is not needed, something simple and low effort is probably best. -- Rick C. + Get 1,000 miles of free Supercharging + Tesla referral code - https://ts.la/richard11209
Reply by ●December 1, 20222022-12-01
On 01/12/2022 21:12, John Larkin wrote:> On Sat, 26 Nov 2022 20:34:23 -0800, John Larkin > <jlarkin@highlandSNIPMEtechnology.com> wrote: > >> We use the efinix T20 trion FPGA. >> >> Questions about the config bit streams: >> >> Are they always the same size, or does it depend on how much logic is >> compiled? Would a simple application use less? >> >> Are the streams very compressible? We have done some simple run-length >> coding to greatly reduce the storage requirement for other FPGAs. >> Configs tend to have long runs of 0's. >> >> The T20/256 claims to need 5.4 megabits. I'd like to store the fpga >> config and application code in a Raspberry Pi Pico, which has 2 MB of >> onboard flash. Storing the full config would use about a third of >> that, so reducing that would be useful. > > Here's a T20 bit stream. The length seems to be constant vs functions > coded, but there are enough runs of all 0's that it's probably worth > compressing. > > https://www.dropbox.com/s/vm247lntp78jm20/Efinix_T20_bitstream.hex?dl=0 > > The actual config file will be binary, not hex of course.Quick scan with one of my utilities gives: Filename : \users\martin\downloads\Efinix~1.hex File size = 4071902 Entropy = 1.225 ( max. 5.545 ) States used = 3.40 ( max. 256 ) Zero frequency : 0-9 11-47 58-64 71-255 Most frequent bytes: 48 30 "0" 2198086 10 A ... 1357302 49 31 "1" 98740 52 34 "4" 97072 56 38 "8" 96870 50 32 "2" 94906 54 36 "6" 26994 51 33 "3" 26880 67 43 "C" 26478 57 39 "9" 25500 65 41 "A" 6820 53 35 "5" 5944 The hex file consists mostly of character "0" bytes and linefeeds. Simple run length encoding would compact it a lot. It seems "7","B","D","E","F" are quite rare in these files. The raw binary file obviously won't have the linefeeds and will be only one byte for every three in the ASCII .hex file so about 1.3M. Back of the envelope RLE might get you a ~20x decrease in size. The right compressor and it could be made a lot smaller. If you put up the binary I'll scan that for byte entropy too. -- Regards, Martin Brown
