Forums

Hardware scripting (a better Arduino)

Started by bitrex June 27, 2017
As someone pointed out a while back while the Arduino toolchain seems to 
be very popular for developing hobbyist embedded applications for 
AVR/ARM, but the API and IDE is kind of a mess. The API is a bastardized 
subset of C++ where a lot of things don't work and which most users only 
use to write kludgey, procedural C-like code; you have C-like global 
functions such as "digitalWrite" which under the hood use 40 or 50 
assembly instructions just to write a bit to a GPIO port.

I think it's certainly possible to make something better that allows 
someone who doesn't want to fight with the tools get real work done.

I was thinking that since little uPs are really more like programmable 
hardware rather than general purpose computers, if one's looking to make 
a microcontroller language for the everyman it doesn't really make sense 
to derive it directly from a general purpose systems programming 
language like C/C++.

In modern computer games the graphics and physics engines are written in 
high-performance compiled languages like C/C++. But the whole product 
doesn't use those languages to define its behavior; usually all the 
plot, mechanics, and other stuff that make a video game a game is 
fleshed out in some kind of scripting language at a much higher level of 
abstraction. This allows people other than rockstar systems programmers 
to contribute to the design and avoids needing to recompile the whole 
codebase every time someone decides this enemy spaceship should fire 
purple lasers instead of green.

I think even 8 bit processors with modern architectures are fast enough 
that one could adapt a similar paradigm to writing embedded apps. The 
high-performance stuff, like interfacing with external hardware through 
GPIO can be written in C/C++, in something like the "policy-based 
design" paradigm where you have an abstract type of device, like 
"Display" or "TemperatureSensor" which defines an interface to the basic 
functions any device of that type should be able to do, and then 
"policies" or "plug-ins" which handle the logic requirements of some 
particular type of TemperatureSensor from some manufacturer. If you want 
to change the sensor you don't rewrite the entire codebase, you just 
rewrite the plug-in.

And then the logic for how the external hardware interacts could be 
written in a very abstract stack machine language, similar to Forth or 
PostScript, or a programmable RPN calculator. An example stack machine 
implementation for Arduino I saw would blink an LED on and off like this:

  13 output
  {
    13 digitalToggle 1000 delay
    true
  } while

These scripts aren't compiled, but actually stored as plaintext strings 
(in compressed mnemonic form) in Flash or EEPROM and then interpreted on 
the fly by the stack machine.
You have just re-invented the PLC, maybe you should look into the
existing "Arduino as a PLC" and "Open PLC" projects?
On 6/27/2017 7:07 AM, bitrex wrote:
> I think even 8 bit processors with modern architectures are fast enough that > one could adapt a similar paradigm to writing embedded apps. The > high-performance stuff, like interfacing with external hardware through GPIO > can be written in C/C++, in something like the "policy-based design" paradigm > where you have an abstract type of device, like "Display" or > "TemperatureSensor" which defines an interface to the basic functions any > device of that type should be able to do, and then "policies" or "plug-ins" > which handle the logic requirements of some particular type of > TemperatureSensor from some manufacturer. If you want to change the sensor you > don't rewrite the entire codebase, you just rewrite the plug-in. > > And then the logic for how the external hardware interacts could be written in > a very abstract stack machine language, similar to Forth or PostScript, or a > programmable RPN calculator. An example stack machine implementation for > Arduino I saw would blink an LED on and off like this: > > 13 output > { > 13 digitalToggle 1000 delay > true > } while > > These scripts aren't compiled, but actually stored as plaintext strings (in > compressed mnemonic form) in Flash or EEPROM and then interpreted on the fly by > the stack machine.
Two obvious problems: - strings take up lots more space than "compiled code" (you were complaining about 50 instructions to toggle a GPIO, upthread -- what would your "string equivalent" form be: "13 output true" - how to handle the case AT RUNTIME when a *parse* of the text doesn't map to legitimate executable instructions: "13 uotput true" Its trivial to write a little interpreter that gives you a bit of both worlds without having to develop a full-fledged compiler or task yourself with generating "really efficient" code. The interpreter can then "playback" the tokenized source code to reveal the source without having to STORE the source. I wrote a little bastardized BASIC that ran in a 647180 (predated the richer SoC's available, now) so you could do things like: SPAWN 100 SPAWN 200 STOP 100 STATE = ON 101 LOAD FLASH_TIMER 300ms LAMP STATE WAIT FLASH_TIMER STATE = !STATE GOTO 101 200 PRINT "Please enter your password:" INPUT PASSWORD ... This might "compile" down to <100 bytes (the largest portion of that being the string that was being PRINTed)
On 28/06/17 00:07, bitrex wrote:
> As someone
Me, I believe.
> pointed out a while back while the Arduino toolchain seems to > be very popular for developing hobbyist embedded applications for > AVR/ARM, but the API and IDE is kind of a mess. The API is a bastardized > subset of C++ where a lot of things don't work and which most users only > use to write kludgey, procedural C-like code; you have C-like global > functions such as "digitalWrite" which under the hood use 40 or 50 > assembly instructions just to write a bit to a GPIO port. > > I think it's certainly possible to make something better that allows > someone who doesn't want to fight with the tools get real work done. > > I was thinking that since little uPs are really more like programmable > hardware rather than general purpose computers, if one's looking to make > a microcontroller language for the everyman it doesn't really make sense > to derive it directly from a general purpose systems programming > language like C/C++. > > In modern computer games the graphics and physics engines are written in > high-performance compiled languages like C/C++. But the whole product > doesn't use those languages to define its behavior; usually all the > plot, mechanics, and other stuff that make a video game a game is > fleshed out in some kind of scripting language at a much higher level of > abstraction. This allows people other than rockstar systems programmers > to contribute to the design and avoids needing to recompile the whole > codebase every time someone decides this enemy spaceship should fire > purple lasers instead of green. > > I think even 8 bit processors with modern architectures are fast enough > that one could adapt a similar paradigm to writing embedded apps.
Yes, I agree. I think a lot of systems would be better implemented that way.
>The > high-performance stuff, like interfacing with external hardware through > GPIO can be written in C/C++, in something like the "policy-based > design" paradigm where you have an abstract type of device, like > "Display" or "TemperatureSensor" which defines an interface to the basic > functions any device of that type should be able to do, and then > "policies" or "plug-ins" which handle the logic requirements of some > particular type of TemperatureSensor from some manufacturer. If you want > to change the sensor you don't rewrite the entire codebase, you just > rewrite the plug-in.
That's what the creators of Arduino should have done with their C++ library. C++ is still the right way to build the system you describe.
> And then the logic for how the external hardware interacts could be > written in a very abstract stack machine language, similar to Forth or > PostScript, or a programmable RPN calculator. An example stack machine > implementation for Arduino I saw would blink an LED on and off like this: > > 13 output > { > 13 digitalToggle 1000 delay > true > } while > > These scripts aren't compiled, but actually stored as plaintext strings > (in compressed mnemonic form) in Flash or EEPROM and then interpreted on > the fly by the stack machine.
This is where I diverge. "Compressed mnemonic" means tokenised. It's better to replace each opcode by a direct pointer to the function that implements the opcode, aka a threaded interpreter. You use a pointer register to replace the PC, so execution is just asm("jmp p++"), and the function extracts its parameters using "*p++". Trivial to implement, and very fast on a not-heavily pipelined CPU. Want help writing the compiler? It's pretty straightforward. Propose a sensible syntax (not Basic, and not RPN/Forth!) and I'll look at it. It would probably even integrate easily into the Arduino toolchain, emitting C++ data definitions to be compiled into flash. Clifford Heath.
On 06/27/2017 09:01 PM, Clifford Heath wrote:
> On 28/06/17 00:07, bitrex wrote: >> As someone > > Me, I believe. > >> pointed out a while back while the Arduino toolchain seems to >> be very popular for developing hobbyist embedded applications for >> AVR/ARM, but the API and IDE is kind of a mess. The API is a bastardized >> subset of C++ where a lot of things don't work and which most users only >> use to write kludgey, procedural C-like code; you have C-like global >> functions such as "digitalWrite" which under the hood use 40 or 50 >> assembly instructions just to write a bit to a GPIO port. >> >> I think it's certainly possible to make something better that allows >> someone who doesn't want to fight with the tools get real work done. >> >> I was thinking that since little uPs are really more like programmable >> hardware rather than general purpose computers, if one's looking to make >> a microcontroller language for the everyman it doesn't really make sense >> to derive it directly from a general purpose systems programming >> language like C/C++. >> >> In modern computer games the graphics and physics engines are written in >> high-performance compiled languages like C/C++. But the whole product >> doesn't use those languages to define its behavior; usually all the >> plot, mechanics, and other stuff that make a video game a game is >> fleshed out in some kind of scripting language at a much higher level of >> abstraction. This allows people other than rockstar systems programmers >> to contribute to the design and avoids needing to recompile the whole >> codebase every time someone decides this enemy spaceship should fire >> purple lasers instead of green. >> >> I think even 8 bit processors with modern architectures are fast enough >> that one could adapt a similar paradigm to writing embedded apps. > > Yes, I agree. I think a lot of systems would be better implemented > that way.
I haven't had a chance to follow up to each reply to this thread individually today, so I'll just use the opportunity to address a couple of the objections "Rob" and "Don Y" mentioned here...the performance of the stack machine doesn't have to be particularly great, as profiling usually shows that most apps spend the vast majority of their time grinding thru the same small bit of code, perhaps only 5% of the total codebase. In an embedded app this might be say some signal processing or other data-cruching code, or handling and dispatching packets coming in over a network. Like in the games analogy, that area is the equivalent of the "engine" that's doing the graphics, and would certainly be at least implemented in C/C++. It will get its work done nearly as fast if the algorithm is simply referenced by some function pointer on a software stack that gets called with other stack objects popped into it, as that's essentially how calling functions on local variables from procedural code works with the intrinsic processor stack defined by the uP's ISA which a C/C++ compiler uses, and with a simple architecture that lacks virtual memoy, cache, TLBs, and all that stuff a well-implemented software stack should be nearly as fast. Much of a codebase wont be performance-critical and using C++ for the rest of it is the proverbial using a sledgehammer to swat a fly.
>> The >> high-performance stuff, like interfacing with external hardware through >> GPIO can be written in C/C++, in something like the "policy-based >> design" paradigm where you have an abstract type of device, like >> "Display" or "TemperatureSensor" which defines an interface to the basic >> functions any device of that type should be able to do, and then >> "policies" or "plug-ins" which handle the logic requirements of some >> particular type of TemperatureSensor from some manufacturer. If you want >> to change the sensor you don't rewrite the entire codebase, you just >> rewrite the plug-in. > > That's what the creators of Arduino should have done with their C++ > library. C++ is still the right way to build the system you describe.
I think part of the reason the asm generated for many Arduino API calls is so bloated is that while it is derived from C++, they made virtually no use of the template metaprogramming functionality available in its modern incarnations and so there have to be tons of overloads, virtual methods, etc. for different targets and user configuration options where the compiler has no idea what is needed and what isn't, so has to toss in the kitchen sink. Template metaprogramming is too abstruse for newbs, but ideally all this platform-specific and configuration stuff would be templated and handled at compile time. The goal of modern C++ is to have "zero overhead abstraction" via generic programming; done correctly most of the important stuff about the target should be known at compile time such that you can certainly use an abstract function like "digitalWrite" in your code, but when its compiled the compiler "knows" that it can smoosh all that abstract "overhead" code down into a single asm bit flip instruction on the proper port for the target device.
>> And then the logic for how the external hardware interacts could be >> written in a very abstract stack machine language, similar to Forth or >> PostScript, or a programmable RPN calculator. An example stack machine >> implementation for Arduino I saw would blink an LED on and off like this: >> >> 13 output >> { >> 13 digitalToggle 1000 delay >> true >> } while >> >> These scripts aren't compiled, but actually stored as plaintext strings >> (in compressed mnemonic form) in Flash or EEPROM and then interpreted on >> the fly by the stack machine. > > This is where I diverge. "Compressed mnemonic" means tokenised. It's > better to replace each opcode by a direct pointer to the function > that implements the opcode, aka a threaded interpreter. You use a > pointer register to replace the PC, so execution is just asm("jmp p++"), > and the function extracts its parameters using "*p++". Trivial to > implement, and very fast on a not-heavily pipelined CPU.
Yeah, you could have each "opcode" represented by a single unsigned char, so with an RPN-kind of language the "interpreter" works pretty much the same as you describe, a program counter stepping thru the script and pushing and popping functions and data on and off the stack, and harvesting return values, as appropriate. You can have foolproof variant types that will happily hold both function pointers and data in the same "package", on the same software stack, using a custom stack allocator designed as appropriate for the capabilities of the hardware. You can have locally scoped lambda functions generated on-the-fly at compile time and stored in flash memory that can be templated to accept any number of arguments from the stack as is required automatically. All these things are standard stuff it the modern C++ hotness, and compile down to having not much more overhead than pure C or asm.
> Want help writing the compiler? It's pretty straightforward. Propose > a sensible syntax (not Basic, and not RPN/Forth!) and I'll look at it. > It would probably even integrate easily into the Arduino toolchain, > emitting C++ data definitions to be compiled into flash. > > Clifford Heath.
This project would have to be pretty low on my list of priorities, unfortunately, but I'll definitely keep the offer in mind...:-)
On 27/06/17 15:07, bitrex wrote:

> I was thinking that since little uPs are really more like programmable hardware > rather than general purpose computers, if one's looking to make a > microcontroller language for the everyman it doesn't really make sense to derive > it directly from a general purpose systems programming language like C/C++.
There is a general philosophical question: is it better to create - a domain-specific language or - a domain-specific library in a standard language Without exception, all the "let's allow easy scripting" (before or after product delivery) using our own DSLanguage have speedily end up being a disgusting mess that even the /creators/ no longer understand. I've seen two at close quarters, and narrowly avoided creating one myself. Even if a DSLanguage isn't an unholy mess, nobody else will want to work with it, there is zero tool support, unless you create it yourself, and you'll have to train people to use and extend it. OTOH a DSLibrary comes with all the usual tool support, people /expect/ to use DSLibraries, and the training and support are the essential minimum. The exception to DSLanguages being universally crap is that well-defined /model/ based languages based on standard concepts can sometimes be beneficial. Classic examples are those based on FSMs and - particularly relevant in this case - ladder logic.
> And then the logic for how the external hardware interacts could be written in a > very abstract stack machine language, similar to Forth or PostScript, or a > programmable RPN calculator.
Stick to Forth rather than invent a manky DSLanguage.
On 6/28/2017 8:08 AM, bitrex wrote:
> I haven't had a chance to follow up to each reply to this thread individually > today, so I'll just use the opportunity to address a couple of the objections > "Rob" and "Don Y" mentioned here...the performance of the stack machine doesn't > have to be particularly great, as profiling usually shows that most apps spend > the vast majority of their time grinding thru the same small bit of code, > perhaps only 5% of the total codebase.
Its not the amount/percentage of time that is spent "less efficiently" (avoiding the term "INefficiently") processing the code but, rather, WHICH code is processed "less efficiently". I found that in the designs that I did (e.g., the '7180 mentioned), I still had to manually code a stub ISR to handle the issues where timeliness was important -- because the interpreted code couldn't guarantee that it would get around to executing in the timeframe available. OTOH, the stubs could invoke code written in this bogo-language -- *if* they were otherwise constrained in frequency, etc. [Note part of this was due to the way I implemented the multitasking executive to expressly eliminate the need for explicit atomic regions and synchronization primitives. Unlike a preemptive context switch, you were largely at the mercy of whatever code happened to be executing at that instant.]
> In an embedded app this might be say > some signal processing or other data-cruching code, or handling and dispatching > packets coming in over a network. Like in the games analogy, that area is the > equivalent of the "engine" that's doing the graphics, and would certainly be at > least implemented in C/C++. > > It will get its work done nearly as fast if the algorithm is simply referenced > by some function pointer on a software stack that gets called with other stack > objects popped into it, as that's essentially how calling functions on local > variables from procedural code works with the intrinsic processor stack defined > by the uP's ISA which a C/C++ compiler uses, and with a simple architecture > that lacks virtual memoy, cache, TLBs, and all that stuff a well-implemented > software stack should be nearly as fast.
This is the approach I've taken with my current project: all of the time/performance-critical services are written in C/ASM while the higher levels of abstraction are written in a scripting language. The latter deprives the developer of the ability to fret over performance issues and concentrate, instead, on what he wants to get *done*. E.g., its easy to see how the "lamp flashing" code presented earlier can be reworked to provide a timeout on the data requested from the user by the prompt (PRINT) that followed in that example. Does the developer really care if each iteration of the FLASH loop is exactly 500ms? Or, that they are all consistent? (obviously, you could write such a loop where that was a primary goal -- but, it typically won't be so why worry about those details of minimizing jitter, etc.?)
> Much of a codebase wont be performance-critical and using C++ for the rest of > it is the proverbial using a sledgehammer to swat a fly.
>>> And then the logic for how the external hardware interacts could be >>> written in a very abstract stack machine language, similar to Forth or >>> PostScript, or a programmable RPN calculator. An example stack machine >>> implementation for Arduino I saw would blink an LED on and off like this: >>> >>> 13 output >>> { >>> 13 digitalToggle 1000 delay >>> true >>> } while >>> >>> These scripts aren't compiled, but actually stored as plaintext strings >>> (in compressed mnemonic form) in Flash or EEPROM and then interpreted on >>> the fly by the stack machine. >> >> This is where I diverge. "Compressed mnemonic" means tokenised. It's >> better to replace each opcode by a direct pointer to the function >> that implements the opcode, aka a threaded interpreter. You use a >> pointer register to replace the PC, so execution is just asm("jmp p++"), >> and the function extracts its parameters using "*p++". Trivial to >> implement, and very fast on a not-heavily pipelined CPU. > > Yeah, you could have each "opcode" represented by a single unsigned char, so > with an RPN-kind of language the "interpreter" works pretty much the same as > you describe, a program counter stepping thru the script and pushing and > popping functions and data on and off the stack, and harvesting return values, > as appropriate.
Exactly. You use the fetched "bytecode" as an index into a list of function pointers and invoke the function referenced. To it, you pass a pointer into the "instruction stream" so it can fetch "arguments" from the instruction stream, returning the updated "PC" for use by the next iteration of the interpreter.
> You can have foolproof variant types that will happily hold both function > pointers and data in the same "package", on the same software stack, using a > custom stack allocator designed as appropriate for the capabilities of the > hardware. You can have locally scoped lambda functions generated on-the-fly at > compile time and stored in flash memory that can be templated to accept any > number of arguments from the stack as is required automatically.
"Typing" is something that requires careful thought. Often, you don't *need* a variety of types: a numeric type and a string type can cover a lot of applications. More types inevitably lead to more conversions between types, formatted printing, etc. My scripting language uses one of each and allows them to "grow", as dictated by the application. The advantage, there, is that the user need not be concerned with things like overflow or ordering operations to maximize preserved precision, etc. The disadvantage is that you do a lot *for* them and take away a fair bit of control that they might want to exert on the performance of their code. [It also means you tend to need runtime GC -- which brings up another variable in performance (as well as more implementation decisions)] At the other end of the spectrum, you can constrain the language to integer/fixed-point numerics and preallocated strings (buffers).
> All these things are standard stuff it the modern C++ hotness, and compile down > to having not much more overhead than pure C or asm.
IME, the bigger issue was having a "development environment" that was easy to explain to others. As I typically am involved in "proof of concept" efforts, I want to be able to show something close to pseudo-code to the client and the folks who will ultimately be charged with bringing the product to manufacturing and then, the market. I don't want to have to "translate" an application back into a *less* capable expressive form than the prototype implementation just to show what it's doing (in detail). I'd rather present the prototype in a more abstracted language and let them refine the efficiency of the implementation to suit their goals (resources, timeliness, etc.) As such, the language wants to be intuitive and not necessarily complex, even if this comes at the expense of some functionality (e.g., I can craft far more efficient code if I have hooks into the MTOS, use of pointer variables, etc. -- but this makes the resulting code look less like "generic pseudocode" and more like a particular implementation). Consider the audience (Arduino) and what they're likely to call upon their "boxes" to do. Will they be scanning raw video from a reflective photodetector to decide barcodes? Or, accepting decoded barcodes (from something) and manipulating them at some higher level...?
On 06/28/2017 01:51 PM, Don Y wrote:

> Exactly. You use the fetched "bytecode" as an index into a list of > function pointers and invoke the function referenced. To it, you pass > a pointer into the "instruction stream" so it can fetch "arguments" from > the instruction stream, returning the updated "PC" for use by the next > iteration of the interpreter. > >> You can have foolproof variant types that will happily hold both >> function pointers and data in the same "package", on the same software >> stack, using a custom stack allocator designed as appropriate for the >> capabilities of the hardware. You can have locally scoped lambda >> functions generated on-the-fly at compile time and stored in flash >> memory that can be templated to accept any number of arguments from >> the stack as is required automatically. > > "Typing" is something that requires careful thought. Often, you don't > *need* > a variety of types: a numeric type and a string type can cover a lot of > applications. More types inevitably lead to more conversions between > types, > formatted printing, etc. My scripting language uses one of each and allows > them to "grow", as dictated by the application.
The C++ solutions available for discriminated unions on x86 platforms are STL containers like boost::variant and boost::any. boost::variant uses function pointers under the hood IIRC, while boost::any uses black magic and is much more generic (and about a thousand times slower.) <http://www.boost.org/doc/libs/1_64_0/doc/html/variant.html> Functions have never been first-class objects in C++ but with C++11 and some template library stuff they can come pretty close. You can pass them around to other functions by reference or value, hold them in containers, put them on stacks and in queues, whatever. I think numeric types should be "strong" and suited to the application, basically classes. Stuff like "int" and "float" are really too broad. Ideally it should never be possible to try to assign the return value of something like a temperature sensor to another type that can hold say complex numbers or floating point values way outside the range of Earthlike temperatures - the compiler should complain. That is to say if one has to convert between different types with any regularity it's probably indicative of bad design.
> The advantage, there, is that the user need not be concerned with things > like overflow or ordering operations to maximize preserved precision, etc. > > The disadvantage is that you do a lot *for* them and take away a fair > bit of control that they might want to exert on the performance of their > code. > > [It also means you tend to need runtime GC -- which brings up another > variable in performance (as well as more implementation decisions)]
A pure functional language implemented via a stack machine doesn't need a garbage collector, as there's no "garbage" to collect. Downside is you do lose flexibility and performance. My gimmick was to try to isolate the mutable state (i.e. when interacting with external widgets through policies written in C++) within APIs that aren't directly accessible as much as possible. Indeed I don't think a pure functional language that never had to take any input or provide any output to the real world would require any mutable state at all.
> As such, the language wants to be intuitive and not necessarily complex, > even if this comes at the expense of some functionality (e.g., I can > craft far more efficient code if I have hooks into the MTOS, use of pointer > variables, etc. -- but this makes the resulting code look less like > "generic pseudocode" and more like a particular implementation). > > Consider the audience (Arduino) and what they're likely to call upon their > "boxes" to do. Will they be scanning raw video from a reflective > photodetector > to decide barcodes? Or, accepting decoded barcodes (from something) and > manipulating them at some higher level...?
Who knows exactly, but they usually want to interact with the real world and other external hardware in some way or another. As it stands at the moment they either have to write the "device driver" themselves or hope some implementation provided by the manufacturer works for their implementation. In my proposal they have to...write the "device driver" themselves or hope some implementation provided by the manufacturer works in their implementation. Seems the least one could do is try to make the rest of the development process bot just as painful...
On 6/28/2017 12:56 PM, bitrex wrote:
> On 06/28/2017 01:51 PM, Don Y wrote: > >> Exactly. You use the fetched "bytecode" as an index into a list of >> function pointers and invoke the function referenced. To it, you pass >> a pointer into the "instruction stream" so it can fetch "arguments" from >> the instruction stream, returning the updated "PC" for use by the next >> iteration of the interpreter. >> >>> You can have foolproof variant types that will happily hold both function >>> pointers and data in the same "package", on the same software stack, using a >>> custom stack allocator designed as appropriate for the capabilities of the >>> hardware. You can have locally scoped lambda functions generated on-the-fly >>> at compile time and stored in flash memory that can be templated to accept >>> any number of arguments from the stack as is required automatically. >> >> "Typing" is something that requires careful thought. Often, you don't *need* >> a variety of types: a numeric type and a string type can cover a lot of >> applications. More types inevitably lead to more conversions between types, >> formatted printing, etc. My scripting language uses one of each and allows >> them to "grow", as dictated by the application. > > The C++ solutions available for discriminated unions on x86 platforms are STL > containers like boost::variant and boost::any. boost::variant uses function > pointers under the hood IIRC, while boost::any uses black magic and is much > more generic (and about a thousand times slower.) > > <http://www.boost.org/doc/libs/1_64_0/doc/html/variant.html> > > Functions have never been first-class objects in C++ but with C++11 and some > template library stuff they can come pretty close. You can pass them around to > other functions by reference or value, hold them in containers, put them on > stacks and in queues, whatever. > > I think numeric types should be "strong" and suited to the application, > basically classes. Stuff like "int" and "float" are really too broad. Ideally > it should never be possible to try to assign the return value of something like > a temperature sensor to another type that can hold say complex numbers or > floating point values way outside the range of Earthlike temperatures - the > compiler should complain.
Again, it depends on the "market" you're trying to address. I have ONE numeric type and one string type. I don't require the user to set aside buffer space for strings -- there are no buffer overruns (until you run out of memory and the system ABENDS.)
> That is to say if one has to convert between different types with any > regularity it's probably indicative of bad design.
How fine-grained do you approach this? Do you define a "foot", type and an "inch" type (derived from "distance")? And prevent them from being combined with volumetric types or units of time? (e.g., miles/hours) Do you require a cast when using an integer iterator to scale a float? (pay = hours * wage_rate) Years ago, I defined a set of classes that let me mix and match dimensioned quantities. It was more work than it was worth. Do you consider printing a numeric value to be a "conversion" operation? Or, scanning a string representation of a numeric value into a numeric data type?
>> The advantage, there, is that the user need not be concerned with things >> like overflow or ordering operations to maximize preserved precision, etc. >> >> The disadvantage is that you do a lot *for* them and take away a fair >> bit of control that they might want to exert on the performance of their >> code. >> >> [It also means you tend to need runtime GC -- which brings up another >> variable in performance (as well as more implementation decisions)] > > A pure functional language implemented via a stack machine doesn't need a > garbage collector, as there's no "garbage" to collect. Downside is you do lose > flexibility and performance.
You lose the ability to deal with mutable objects of varying sizes (e.g., strings) without forcing the user to do the housekeeping (prone to error). Likewise, you require all resources to be persistent -- you can't spawn/fork new tasks/processes and kill off old ones that have served their purposes.
> My gimmick was to try to isolate the mutable state (i.e. when interacting with > external widgets through policies written in C++) within APIs that aren't > directly accessible as much as possible. > > Indeed I don't think a pure functional language that never had to take any > input or provide any output to the real world would require any mutable state > at all. > >> As such, the language wants to be intuitive and not necessarily complex, >> even if this comes at the expense of some functionality (e.g., I can >> craft far more efficient code if I have hooks into the MTOS, use of pointer >> variables, etc. -- but this makes the resulting code look less like >> "generic pseudocode" and more like a particular implementation). >> >> Consider the audience (Arduino) and what they're likely to call upon their >> "boxes" to do. Will they be scanning raw video from a reflective photodetector >> to decide barcodes? Or, accepting decoded barcodes (from something) and >> manipulating them at some higher level...? > > Who knows exactly, but they usually want to interact with the real world and > other external hardware in some way or another. As it stands at the moment they > either have to write the "device driver" themselves or hope some implementation > provided by the manufacturer works for their implementation.
"In theory", one can lift most drivers from other places and rework them to fit the specific needs of the application at hand. The problem lies with folks who want to pollute the drivers with stuff that doesn't belong there; usually because of timeliness issues or lack of understanding as to how to walk and chew gum at the same time. I.e., a better approach (IMO) is to put in place mechanisms that address THESE issues as "builtins" -- providing a framework on which to drape the application. E.g., even in tiny applications, I support STDIN, STDOUT and STDERR for each task/process. So, a developer doesn't have to invent his own way of getting data into and out-of a process. Less obvious is this also magically manages the resource sharing behind the scenes. So, I can define STDERR to be the "debugger port" and know that I will see a stream of messages like: task0: starting up task5: waiting for input task3: terminating ... instead of: tttask3aas:k5: s...
> In my proposal they have to...write the "device driver" themselves or hope some > implementation provided by the manufacturer works in their implementation. > Seems the least one could do is try to make the rest of the development process > bot just as painful...
I want a language (targeting a particular type of applications) to deal with the "little crap" that will invariably eat my lunch. While the developer might pick a suitable size data type for height, width, length and duration. But, will the calculations that he has undertaken ever overflow in the process of computin the final result (volume/unit time)? While sleep() lets me pause a task/process, it does little to help me if I want to mark a point in the execution stream and check to see if a particular interval has expired at some OTHER point in the execution stream. Or, "signal" me when that time period expires, regardless of what I might be doing at the time. Should I have to take explicit measures to ensure atomic access to complex variables/structures? Should I have to understand the mechanisms to spawn (fork/exec) a new task? And, pass information between it and myself (child/parent)? Should I have to understand the mechanics of IPC/RPC in order to be able to invoke a "foreign" process/method? If these are "optional" uses to which a language is applied, then it makes sense to put them in an OPTIONAL library. OTOH, if the language is intended for use in a particular class of applications, then putting them in a library just makes for a messier implementation. I.e., if I a communication channel (a construct used in Limbo/Inferno) happens to be moved to a remote node in a refactoring, should I have to rewrite the code that accesses it -- to invoke an RPC instead of just referencing it "as if" completely local?
On 29/06/17 01:08, bitrex wrote:
> On 06/27/2017 09:01 PM, Clifford Heath wrote: >> On 28/06/17 00:07, bitrex wrote: >>> The >>> high-performance stuff, like interfacing with external hardware through >>> GPIO can be written in C/C++, in something like the "policy-based >>> design" paradigm where you have an abstract type of device, like >>> "Display" or "TemperatureSensor" which defines an interface to the basic >>> functions any device of that type should be able to do, and then >>> "policies" or "plug-ins" which handle the logic requirements of some >>> particular type of TemperatureSensor from some manufacturer. If you want >>> to change the sensor you don't rewrite the entire codebase, you just >>> rewrite the plug-in. >> >> That's what the creators of Arduino should have done with their C++ >> library. C++ is still the right way to build the system you describe. > > I think part of the reason the asm generated for many Arduino API calls > is so bloated is that while it is derived from C++, they made virtually > no use of the template metaprogramming functionality available in its > modern incarnations and so there have to be tons of overloads, virtual > methods, etc. for different targets and user configuration options where > the compiler has no idea what is needed and what isn't, so has to toss > in the kitchen sink.
Exactly what I've started doing. E.g. <https://github.com/cjheath/TSEvents>, <https://github.com/cjheath/AD9851> The AD9851 template has six parameters which are compile-time constants, but would otherwise have to be stored in instance variables. Similarly with my port-pin templates, posted here a month or three back.
> Template metaprogramming is too abstruse for newbs,
newbs don't really need to know how to write it, just how to use it. You do wind up with some ugly hacks (like empty/non-empty base classes in templates, if e.g. you want to make that AD9851 calibration to be adjustable at runtime). And some of those hacks become virtual functions when you don't really need them. C++ is an ugly language full of warts.
> but ideally all this > platform-specific and configuration stuff would be templated and handled > at compile time. The goal of modern C++ is to have "zero overhead > abstraction" via generic programming; done correctly most of the > important stuff about the target should be known at compile time such > that you can certainly use an abstract function like "digitalWrite" in > your code, but when its compiled the compiler "knows" that it can smoosh > all that abstract "overhead" code down into a single asm bit flip > instruction on the proper port for the target device.
Exactly what my port pin templates do. Pity I have to rewrite every library I want to use, but it does make them faster and smaller.
>> This is where I diverge. "Compressed mnemonic" means tokenised. > Yeah, you could have each "opcode" represented by a single unsigned > char,
Except then you get a jump-table interpreter which kills branch prediction so you tend to flush the pipeline on every instruction. That doesn't matter on an AVR, but does on Cortex. The advantage with threaded interpreters is the opcode *is* the pointer to the implementation. Not as good as JIT, but still.
> All these things are standard stuff it the modern C++ hotness, and > compile down to having not much more overhead than pure C or asm.
Yes. It's esoteric, but doesn't have to be hard for end-users. Clifford Heath.