On Wednesday, January 30, 2019 at 7:22:00 AM UTC-8, gnuarm.del...@gmail.com wrote:
> On Wednesday, January 30, 2019 at 9:44:05 AM UTC-5, 69883925...@nospam.org wrote:
> > You stated that sequentially writing to FLASH was dangerous,
> I've never said that. My point is that Flash is not reliable. Rely on it at your own risk.
Don't believe such an unqualified statement. Flash has limited erase/rewrite
cycles (and humans have limited lifetime, too). But 'reliable' depends on
usage, and a WORM disk (write-once-read-mainly) is very useful, and
exemplifies good practices for Flash devices. WORM disk is good for 1 write cycle.
Flash devices are good for ten-thousand-to-millions. Moore's law would
suggest Flash is decades more advanced than WORM.
If used like in a camera (write but usually not overwrite when picture taking,
erase only when the user takes action to free space) Flash is totally reliable.
Until the year 2300, my few-days-between handling of the useful
snapshot device won't expose me to any real risk.
Some filesystem actions, though, like defragmenting, journaling, registry
maintenance, will inevitably overwrite in a thoughtless and uncontrolled
manner. Designing a Flash disk to look like a rotating-rust disk was a perilous
decision, and few if any OS controls were really ready for the shift.
So, a busy data center might easily see Flash devices failing, because of
the nature of the traffic through those disks. A savvy use of the resource
can avoid the problem, and let's hope the walled enclaves of OS
gnomes are aware and active enough to fit the Flash resource into our
boxes without overstressing its minor weaknesses.
Reply by ●January 30, 20192019-01-30
<698839253X6D445TD@nospam.org> wrote in news:q2slt3$emn$1
@gioia.aioe.org:
> For SDcards the bad blocks management is in the card.
>
>
Exactly... and that ALWAYS invisible to even the 'file system' placed
within a 'volume' partitioned onto the 'drive'. The internals always
have space set aside for re-assignment of storage space such that from
the perspective of a given file system, there is no such event as a bad
write or 'bad block' to be bypassed or circumnavigated. There is also
no such animal as file fragmentation. Incrementally updated files
always get full rewrites of the entire file or the space for the
incremental update is contiguous with the existing file location.
Reply by ●January 30, 20192019-01-30
gnuarm wrote
>On Wednesday, January 30, 2019 at 12:12:11 PM UTC-5, 69883925...@nospam.org wrote:
>> On a sunny day (Wed, 30 Jan 2019 07:21:55 -0800 (PST)) it happened
>> gnuarm wrote
>> >On Wednesday, January 30, 2019 at 9:44:05 AM UTC-5, 69883925...@nospam.org wrote:
>> >>
>> >> You stated that sequentially writing to FLASH was dangerous,
>> >> I hope I made it clear to you and everybody else that is bull.
>> >> Every OS also does that.
>> >
>> >I've never said that.
>>
>> <quote>
>> >>>> but your idea of dedicating a sector per data record
>> >>>> is poor in the extreme. Sectors on Flash are not very reliable. It is a
>> >>>> good idea to have a flash file system to manage the good/bad blocks for you.
>> >>>> I guess you can do that yourself, but are you thinking of that?
>> <end quote>
>>
>>
>> You neither understood when writing that how an OS works, nor how the SDcard's internals work.
>> I hope you do now, else read that link again:
>>
>> https://wiki.linaro.org/WorkingGroups/KernelArchived/Projects/FlashCardSurvey?action=show&redirect=WorkingGroups%2FKernel%2FProjects%2FFlashCardSurvey
>> For SDcards the bad blocks management is in the card.
>
>You are making my case. I didn't say anything about "sequentially writing to FLASH was dangerous".
>
>I didn't say anything about how an "operating system" works. Or do I misunderstand what "OS" means? As usual, you read what
>you want to read.
>
>Try opening your mind to understand what I meant, not what you want my words to mean.
Read your own text that I quoted again.
Better even study the link.
Reply by ●January 30, 20192019-01-30
On Wednesday, January 30, 2019 at 12:12:11 PM UTC-5, 69883925...@nospam.org wrote:
> On a sunny day (Wed, 30 Jan 2019 07:21:55 -0800 (PST)) it happened
> gnuarm wrote
> >On Wednesday, January 30, 2019 at 9:44:05 AM UTC-5, 69883925...@nospam.org wrote:
> >>
> >> You stated that sequentially writing to FLASH was dangerous,
> >> I hope I made it clear to you and everybody else that is bull.
> >> Every OS also does that.
> >
> >I've never said that.
>
> <quote>
> >>>> but your idea of dedicating a sector per data record
> >>>> is poor in the extreme. Sectors on Flash are not very reliable. It is a
> >>>> good idea to have a flash file system to manage the good/bad blocks for you.
> >>>> I guess you can do that yourself, but are you thinking of that?
> <end quote>
>
>
> You neither understood when writing that how an OS works, nor how the SDcard's internals work.
> I hope you do now, else read that link again:
> https://wiki.linaro.org/WorkingGroups/KernelArchived/Projects/FlashCardSurvey?action=show&redirect=WorkingGroups%2FKernel%2FProjects%2FFlashCardSurvey
> For SDcards the bad blocks management is in the card.
You are making my case. I didn't say anything about "sequentially writing to FLASH was dangerous".
I didn't say anything about how an "operating system" works. Or do I misunderstand what "OS" means? As usual, you read what you want to read.
Try opening your mind to understand what I meant, not what you want my words to mean.
Rick C.
++- Get 6 months of free supercharging
++- Tesla referral code - https://ts.la/richard11209
Reply by ●January 30, 20192019-01-30
On a sunny day (Wed, 30 Jan 2019 07:21:55 -0800 (PST)) it happened
gnuarm wrote
>On Wednesday, January 30, 2019 at 9:44:05 AM UTC-5, 69883925...@nospam.org wrote:
>>
>> You stated that sequentially writing to FLASH was dangerous,
>> I hope I made it clear to you and everybody else that is bull.
>> Every OS also does that.
>
>I've never said that.
<quote>
>>>> but your idea of dedicating a sector per data record
>>>> is poor in the extreme. Sectors on Flash are not very reliable. It is a
>>>> good idea to have a flash file system to manage the good/bad blocks for you.
>>>> I guess you can do that yourself, but are you thinking of that?
On Wednesday, January 30, 2019 at 9:44:05 AM UTC-5, 69883925...@nospam.org wrote:
>
> You stated that sequentially writing to FLASH was dangerous,
> I hope I made it clear to you and everybody else that is bull.
> Every OS also does that.
I've never said that. My point is that Flash is not reliable. Rely on it at your own risk.
> It's simple.
Yes, it is.
Rick C.
+-+ Get 6 months of free supercharging
+-+ Tesla referral code - https://ts.la/richard11209
Reply by ●January 30, 20192019-01-30
gnuarm wrote
>I don't have a lot of evidence to offer showing the failure rates among flash
>devices, but I have personally had flash devices fail and I have read many,
>many warnings that flash devices have a relatively high failure rate and
>should not be used as solitary backup media for important information. Much
>of that has been here in this group.
Well, I am not much into arguing for the sake of arguing.
Sure there are many warnings for everything, including glowballworming.
I have had a harddisk fail simply because I dropped those,
the FLASH based memory cards dropped many times did survive.
I have seen optical media fail on write (I always run a compare against the source),
likely dust particles, and older ones that had mold spots in those that I returned.
I have seen writes to SDcard with 'dd' fail on verification in read-back,
but to many other 'on the border' things happening to blame the card (card images are a different matter).
I have never seen a data record fail to SDcard in my designs.
That all means very little, but the mechanical strength, weight, speed (no seek times), low power use,
makes these cards a first choice between those media.
You stated that sequentially writing to FLASH was dangerous,
I hope I made it clear to you and everybody else that is bull.
Every OS also does that.
It's simple.
Reply by ●January 30, 20192019-01-30
On Wednesday, January 30, 2019 at 3:31:29 AM UTC-5, 69883925...@nospam.org wrote:
> gnuarm wrote
> >On Tuesday, January 29, 2019 at 3:59:12 PM UTC-5, 69883925...@nospam.org wrote:
> >> >"Bad sector handling" doesn't prevent all data loss. Flash memory is particularly
> >>
> >>unreliable perhaps only better than spinning rust. Since sectors can
> >> >be lost putting a record within a single sector is not a very reliable way
> >>
> >>to prevent data loss.
> >>
> >> See the link I gave:
> >>
> >> https://wiki.linaro.org/WorkingGroups/KernelArchived/Projects/FlashCardSurvey?action=3Dshow&redirect=3DWorkingGroups%2FKernel%2FProjects%2FFlashCardSurvey
> >
> >I
> >don't see anything on error recovery or failure rates.
>
> Internally in the card a failed write to a physical sector results in that sector marked as 'bad' and the
> translation table from logical sector to physical sector being updated (also in flash in that card)
> to point to a good sector that is then used for write.
> If this process fails you get a write error.
> Write returning OK is data OK on the card.
> In case of data logging, writing sector after sector, or block after block (whatever is best for the card)
> is no different from sequentially writing via some OS using some filesystem.
> I have inspected and recovered data (images and video) from cards that were erased (report was in sci.crypt)
> found all was neatly sequentially written, even in case of that MS OS.
> Fragmentation only occurs after many read writes erasures of files of different length by the OS.
> So why use a filesystem if you only have one data stream.
> Cards can go bad, use a hammer and nail, heat those, saw, maybe big sparks (have not tried), EMP,
> to either physically damage the on board controller or cause enough charge change in the storage cells.
>
> For the rest you can drop those, shake those, unlike harddisks,
If the errors are well behaved, then they will be mitigated. If a sector is not bad at the time of writing or the error is not apparent when written, but then an error develops when it is read, this does not mitigate that problem.
You keep talking about file systems. I've never said anything about file systems. I've simply pointed out that Flash memory is not very reliable compared to the rest of a properly designed electronic system. I've also said nothing that is particular to SD cards.
> >> I record a lot on FLASH, and now I am talking about SDcards, and if you check
> >write call error return you will know
> >> when a card is defective, or simply full.
> >> And look at that link, not all cards are the same and allow the same random
> >access.
> >
> >So?
> >
> >
> >> Think about it: All those digital cameras, HD recorders, no problem.
> >
> >Where do you get the idea there is "no problem"???
>
> Because there is no problem :-)
>
> I have SDcards in use for 6 years or more all day, cheap ones too, and those still work.
> All old cards from video cameras still work.
> I do make backups on harddisk and optical though, so I can compare.
I don't have a lot of evidence to offer showing the failure rates among flash devices, but I have personally had flash devices fail and I have read many, many warnings that flash devices have a relatively high failure rate and should not be used as solitary backup media for important information. Much of that has been here in this group.
I've also never had a hard drive fail that I can recall. Does that mean I should consider hard drives to be reliable?
> >> As to writing to flash in a micro, I do not do that,
> >> I do not like the idea of boot loaders and sending the executable over the
> >net, encrypted or not.
> >>
> >> Simply hand or mail a new chip.
> >> The BIG advantage of that is, that if things do not work for any reason (and
> >it would not be the first time a software update
> >> introduces new bugs), then you can simply put the old chip back.
> >> No FLASH is really reliable,
> >
> >Agreed on this point.
>
> I think that line by me 'No FLASH is really reliable,' can be interpreted 2 ways, I did mean without the No :-)
>
>
> >> As to dinos are dead, in my view many things can be done faster and more reliable
> >by small micros, or FPGAs then huge
> >> multi tasking multi-cores running Linux.
> >> The software in a few Microchip PICs for my drone an example of that,
> >
> >Horses for courses. The only reason for using an OS is if it provides some
> >functionality that is hard to get any other way. If you want to communicate
> >over wide bandwidth wifi or any of a dozen other interfaces that an OS provides
> >easily, then an OS is justified. Using Linux simply because you can
> >get a processor on the FPGA chip is not a very good reason for using it with
> >LInux.
> >
> >By the same token, many people have a mistaken impression that only MCUs can
> >be small and cheap. Sure, there are no $0.25 FPGAs available in an 8 pin
> >DIP, but for many lower end apps a $2 FPGA works as well if not better than
> >a $2 MCU. It's not uncommon to have some unique interface that is hard to
> >do in an MCU even with all the various peripherals on chip.
>
> I think we agree on the main point, apart from SDcards perhaps, what is your problem with SDcards?
I think you have failed to read properly what I have written and have read much that I did not write.
Rick C.
+-- Get 6 months of free supercharging
+-- Tesla referral code - https://ts.la/richard11209
Internally in the card a failed write to a physical sector results in that sector marked as 'bad' and the
translation table from logical sector to physical sector being updated (also in flash in that card)
to point to a good sector that is then used for write.
If this process fails you get a write error.
Write returning OK is data OK on the card.
In case of data logging, writing sector after sector, or block after block (whatever is best for the card)
is no different from sequentially writing via some OS using some filesystem.
I have inspected and recovered data (images and video) from cards that were erased (report was in sci.crypt)
found all was neatly sequentially written, even in case of that MS OS.
Fragmentation only occurs after many read writes erasures of files of different length by the OS.
So why use a filesystem if you only have one data stream.
Cards can go bad, use a hammer and nail, heat those, saw, maybe big sparks (have not tried), EMP,
to either physically damage the on board controller or cause enough charge change in the storage cells.
For the rest you can drop those, shake those, unlike harddisks,
>> I record a lot on FLASH, and now I am talking about SDcards, and if you check
>write call error return you will know
>> when a card is defective, or simply full.
>> And look at that link, not all cards are the same and allow the same random
>access.
>
>So?
>
>
>> Think about it: All those digital cameras, HD recorders, no problem.
>
>Where do you get the idea there is "no problem"???
Because there is no problem :-)
I have SDcards in use for 6 years or more all day, cheap ones too, and those still work.
All old cards from video cameras still work.
I do make backups on harddisk and optical though, so I can compare.
>> As to writing to flash in a micro, I do not do that,
>> I do not like the idea of boot loaders and sending the executable over the
>net, encrypted or not.
>>
>> Simply hand or mail a new chip.
>> The BIG advantage of that is, that if things do not work for any reason (and
>it would not be the first time a software update
>> introduces new bugs), then you can simply put the old chip back.
>> No FLASH is really reliable,
>
>Agreed on this point.
I think that line by me 'No FLASH is really reliable,' can be interpreted 2 ways, I did mean without the No :-)
>> As to dinos are dead, in my view many things can be done faster and more reliable
>by small micros, or FPGAs then huge
>> multi tasking multi-cores running Linux.
>> The software in a few Microchip PICs for my drone an example of that,
>
>Horses for courses. The only reason for using an OS is if it provides some
>functionality that is hard to get any other way. If you want to communicate
>over wide bandwidth wifi or any of a dozen other interfaces that an OS provides
>easily, then an OS is justified. Using Linux simply because you can
>get a processor on the FPGA chip is not a very good reason for using it with
>LInux.
>
>By the same token, many people have a mistaken impression that only MCUs can
>be small and cheap. Sure, there are no $0.25 FPGAs available in an 8 pin
>DIP, but for many lower end apps a $2 FPGA works as well if not better than
>a $2 MCU. It's not uncommon to have some unique interface that is hard to
>do in an MCU even with all the various peripherals on chip.
I think we agree on the main point, apart from SDcards perhaps, what is your problem with SDcards?
Reply by ●January 29, 20192019-01-29
On Tuesday, January 29, 2019 at 3:59:12 PM UTC-5, 69883925...@nospam.org wrote:
> gnuarm wrote
> >On Sunday, January 27, 2019 at 3:25:42 AM UTC-5, 69883925...@nospam.org wrote:
> >>
> >gnuarm wrote
> >> >Not sure what a Dino is,
> >>
> >> https://en.wikipedia.org/wiki/Dinosaur
> >> dinosaur are extinct
> >> The story goes, because those were so big, that if one was bitten in the tail
> >it took
> >> almost a second for the nerve signal to reach the brain and be processed,
> >> making those an easy victim for other animals and human hunters.
> >
> >Story is wrong though - just like the human brain doesn't need to process an
> >injury to a hand to cause the arm muscles to retract the hand. "Dinos" mostly
> >likely had multiple brains to control the body. I still don't know what
> >you meant by "Dinos are dead". Obviously some sort of comparison that is
> >clear in your mind, but you didn't make any connections, so I have no idea
> >what you are talking about.
>
> Yes there are reflexes, but it (the dino) also needs to take evasive action for example,
> strike back, etc.
> I don't know about multiple brains, I do know the nerve signals for pain are very slow:
> From https://hypertextbook.com/facts/2002/DavidParizh.shtml
> 0.61m/s (pain)
> So if end of tail of a dino is 20 meters from its head, it would not notice the rat ? tiger? whatever
> taking a byte for about 12 seconds! The tiger would easily get away, same for a human hunter.
> Well theory of course, never tried it on a live one...
Likely you are extrapolating beyond your data set.
> >> >but your idea of dedicating a sector per data record
> >> >is poor in the extreme. Sectors on Flash are not very reliable. It is a
> >>
> >>good idea to have a flash file system to manage the good/bad blocks for you.
> >>
> >> I guess you can do that yourself, but are you thinking of that?
> >>
> >> No, bad sector handling is done by the SDcard firmware
> >> we are talking about SDcards for data storage.
> >
> >"Bad sector handling" doesn't prevent all data loss. Flash memory is particularly
> >unreliable perhaps only better than spinning rust. Since sectors can
> >be lost putting a record within a single sector is not a very reliable way
> >to prevent data loss.
>
> See the link I gave:
> https://wiki.linaro.org/WorkingGroups/KernelArchived/Projects/FlashCardSurvey?action=show&redirect=WorkingGroups%2FKernel%2FProjects%2FFlashCardSurvey
I don't see anything on error recovery or failure rates.
> I record a lot on FLASH, and now I am talking about SDcards, and if you check write call error return you will know
> when a card is defective, or simply full.
> And look at that link, not all cards are the same and allow the same random access.
So?
> Think about it: All those digital cameras, HD recorders, no problem.
Where do you get the idea there is "no problem"???
> As to writing to flash in a micro, I do not do that,
> I do not like the idea of boot loaders and sending the executable over the net, encrypted or not.
>
> Simply hand or mail a new chip.
> The BIG advantage of that is, that if things do not work for any reason (and it would not be the first time a software update
> introduces new bugs), then you can simply put the old chip back.
> No FLASH is really reliable,
Agreed on this point.
> As to dinos are dead, in my view many things can be done faster and more reliable by small micros, or FPGAs then huge
> multi tasking multi-cores running Linux.
> The software in a few Microchip PICs for my drone an example of that,
Horses for courses. The only reason for using an OS is if it provides some functionality that is hard to get any other way. If you want to communicate over wide bandwidth wifi or any of a dozen other interfaces that an OS provides easily, then an OS is justified. Using Linux simply because you can get a processor on the FPGA chip is not a very good reason for using it with LInux.
By the same token, many people have a mistaken impression that only MCUs can be small and cheap. Sure, there are no $0.25 FPGAs available in an 8 pin DIP, but for many lower end apps a $2 FPGA works as well if not better than a $2 MCU. It's not uncommon to have some unique interface that is hard to do in an MCU even with all the various peripherals on chip.
Rick C.
-++ Get 6 months of free supercharging
-++ Tesla referral code - https://ts.la/richard11209