Forums

Hardware scripting (a better Arduino)

Started by bitrex June 27, 2017
On 29/06/17 03:51, Don Y wrote:
> On 6/28/2017 8:08 AM, bitrex wrote: >> performance of the stack machine doesn't have to be particularly >> great, as profiling usually shows >>> This is where I diverge. "Compressed mnemonic" means tokenised. It's >>> better to replace each opcode by a direct pointer to the function >>> that implements the opcode, aka a threaded interpreter. >> Yeah, you could have each "opcode" represented by a single unsigned >> char > Exactly. You use the fetched "bytecode" as an index into a list of > function pointers and invoke the function referenced.
Congratulations. Your interpreter technology is now only 2^5 years out of date.
> "Typing" is something that requires careful thought. Often, you don't > *need* a variety of types: a numeric type and a string type can cover a lot of > applications.
Don: typing is not about such data values. That, again, is a view of typing that's 2^5 years out of date. When you talk about types in terms of "string" vs "integer", you display your ignorance of what "type" actually means. Modern typing is about function signatures, and the exhaustive collection of functions that an object responds to. An integer is an integer because of what you can do to/with it, not how you store it. The same goes for strings, LCD displays, and DDS synthesizers.
> The disadvantage is that you do a lot *for* them and take away a fair > bit of control
Sorry, but that all is pure bullshit. Types *give* you control they don't take it away. But you don't seem to know what a type is. Clifford Heath.
On 29/06/17 01:30, Tom Gardner wrote:
> Stick to Forth rather than invent a manky DSLanguage.
You mean, *another* manky DSLanguage, like Forth.
On 6/28/2017 2:04 PM, Don Y wrote:
> On 6/28/2017 12:56 PM, bitrex wrote: >> On 06/28/2017 01:51 PM, Don Y wrote: >> >>> Exactly. You use the fetched "bytecode" as an index into a list of >>> function pointers and invoke the function referenced. To it, you pass >>> a pointer into the "instruction stream" so it can fetch "arguments" from >>> the instruction stream, returning the updated "PC" for use by the next >>> iteration of the interpreter. >>> >>>> You can have foolproof variant types that will happily hold both function >>>> pointers and data in the same "package", on the same software stack, using >>>> a custom stack allocator designed as appropriate for the capabilities of >>>> the hardware. You can have locally scoped lambda functions generated >>>> on-the-fly at compile time and stored in flash memory that can be templated >>>> to accept any number of arguments from the stack as is required automatically. >>> >>> "Typing" is something that requires careful thought. Often, you don't *need* >>> a variety of types: a numeric type and a string type can cover a lot of >>> applications. More types inevitably lead to more conversions between types, >>> formatted printing, etc. My scripting language uses one of each and allows >>> them to "grow", as dictated by the application. >> >> The C++ solutions available for discriminated unions on x86 platforms are STL >> containers like boost::variant and boost::any. boost::variant uses function >> pointers under the hood IIRC, while boost::any uses black magic and is much >> more generic (and about a thousand times slower.) >> >> <http://www.boost.org/doc/libs/1_64_0/doc/html/variant.html> >> >> Functions have never been first-class objects in C++ but with C++11 and some >> template library stuff they can come pretty close. You can pass them around >> to other functions by reference or value, hold them in containers, put them >> on stacks and in queues, whatever. >> >> I think numeric types should be "strong" and suited to the application, >> basically classes. Stuff like "int" and "float" are really too broad. Ideally >> it should never be possible to try to assign the return value of something >> like a temperature sensor to another type that can hold say complex numbers >> or floating point values way outside the range of Earthlike temperatures - >> the compiler should complain. > > Again, it depends on the "market" you're trying to address. I have > ONE numeric type and one string type. I don't require the user to > set aside buffer space for strings -- there are no buffer overruns > (until you run out of memory and the system ABENDS.)
This is the sort of script a "user" might write (with the declarations elided and a fair bit of hand-waving -- just to give a flavor of what a user is likely to WANT to do and the amount of detail he DOESN'T NEED TO SEE). He'd undoubtedly cobble it together from other scripts that *appeared* to work, blissfully ignorant (an intentional consequence!) of all the machinations happening behind the scenes: wait(TOMORROW) wait until the time-of-day indicates a date one greater than when this initially executed (i.e., just after midnight) (error, sun_up) := astronomical_event(here, today, SUNRISE) if (error != nil) ... Declare "error" to be of type "error_t" and "sun_up" to be a "time_of_day_t" (both as defined in the prototype for "astronomical_event()" function). Invoke the astronomical_event RPC passing our current location ("here") and date ("today") to it while requesting the datum called "SUNRISE". Assign any returned error code to "error" and the sought after time to "sun_up". Note that something else has determined where "here" happens to be (for a home/business, it's essentially a constant; for a MOTOR HOME it could *vary* from day to day, hour to hour! The user shouldn't have to concern himself with those sorts of details...) wait(sun_up) twiddle our thumbs until the sun is expected to rise, *today* (bug if we're too far north!) confirmation: channel of (processID, error_code) create a communication channel that can be used to interactively receive data from the processes we're about to spawn -- those data to consist of tuples containing a process identifier and an error_code rooms := query(house_layout, "room", "exposure = east") query the database for a list of rooms that have an eastern exposure count := 0 while (rooms != nil) { room := hd rooms rooms = tl rooms spawn open_blinds(room, confirmation) count++ } peal each east-facing room off the "list of rooms" and create a process for each to open the blinds in that room. Provide each process with a handle to the communication channel so they can report back to us! (note that we have no idea WHERE those processes will actually execute, or even if they will all execute on the same node; but, we *do* know that the comm channel will magically connect each of them to us.) So, we should expect "count" replies! while (count > 0) { (process, error) := <= confirmation if (error != nil) print("OhMiGosh! " process " reported " error) count-- } wait for replies from all of these spawned processes (they will terminate when they've performed their desired actions) -- note that we have no idea what order the processes will post their replies -- nor do we care! (error, sun_down) := astronomical_event(here, today, SUNSET) if (error != nil) ... end_day := sun_down - 15 * MINUTES wait(end_day) we'll close the blinds *before* the sun actually sets so it doesn't come streaming in the west-facing windows as it falls low in the sky. A smarter routine would take into account the extent of the overhang on that side of the house and query when the sun would fall below a certain elevation -- based on our location and the current date -- that would cause the sun's rays to come through those windows! so, similarly issue another query to identify WEST facing rooms and spawn processes to CLOSE the blinds in those rooms at end_day, harvesting all replies by reusing that communication channel (whose endpoint is fixed, HERE), etc. Note that the user doesn't have to be concerned with whether we have 0, 1, 5, or 8-gazillion rooms in the house. Nor does he care about the length of the list returned from these queries. Nor the lengths of the strings used to define each "room". Nor the format of the "here" or "today" 'constants'. He doesn't care that print may not be wired to a "console" or any other interface that the user is likely to see -- that's something else's "problem". Note that the language infers the types of variables from the asignments made to them (e.g., because "confirmation" is defined as a "channel of (processID, error_code)", the types of the "(process, error)" tuple are infered from the assignment *from* that channel. This helps reduce the "declaratory clutter" in most programs. The syntax is unfortunate. But, I've not been able to come up with a cleaner way of expressing the same constructs -- without adding arbitrary function names (like "read_channel()" instead of the "<=" operator). Likewise, the error handling is something that a user would rather not have to deal with -- nor is he likely to handle it all well! An alternative is to catch exceptions but, to the user, that just looks like a glorified ABEND! :< E.g., astronomical_event() could return obvious errors like "invalid place", "invalid date", "unknown event", etc. But, as its an RPC/IPC, it could also return more esoteric errors like "service not available", "insufficient privilege for requested operation", "timeout", etc. There are just some things that you can't elide and the user has to expect that they *might* occur (else his code fails for unspecified reasons). If *I* was writing this (and was at a facility with scads of windows, etc.), I'd prefer to use a cursor to parse the results of the queries "server side" rather than risking pulling some glob of data into my local address space just to nibble away at a SECOND COPY of it (the first residing on the DBMS server!). Or, I might issue a query to count the number of records satisfying the query's constraints *before* actually issuing the query: "Wow! 18,932 windows??! Something is wonky! I sure don't want to set up 18,932 processes -- even if I parse the queries server-side to avoid that overhead -- as that will undoubtedly exceed my resource quota leaving me wondering why the program crashed before it had a chance to get started!" So, let the user be inefficient and hope he improves (or is helped to improve) without forcing him to understand the mechanisms and costs of his actions.
On 29/06/17 09:52, Clifford Heath wrote:
> On 29/06/17 03:51, Don Y wrote: >> On 6/28/2017 8:08 AM, bitrex wrote: >>> performance of the stack machine doesn't have to be particularly >>> great, as profiling usually shows >>>> This is where I diverge. "Compressed mnemonic" means tokenised. It's >>>> better to replace each opcode by a direct pointer to the function >>>> that implements the opcode, aka a threaded interpreter. >>> Yeah, you could have each "opcode" represented by a single unsigned >>> char >> Exactly. You use the fetched "bytecode" as an index into a list of >> function pointers and invoke the function referenced. > > Congratulations. Your interpreter technology is now only 2^5 years > out of date. > >> "Typing" is something that requires careful thought. Often, you don't >> *need* a variety of types: a numeric type and a string type can cover >> a lot of >> applications. > > Don: typing is not about such data values. That, again, is a view > of typing that's 2^5 years out of date. When you talk about types > in terms of "string" vs "integer", you display your ignorance of > what "type" actually means. > > Modern typing is about function signatures, and the exhaustive > collection of functions that an object responds to. An integer is > an integer because of what you can do to/with it, not how you store > it. The same goes for strings, LCD displays, and DDS synthesizers. > >> The disadvantage is that you do a lot *for* them and take away a fair >> bit of control > > Sorry, but that all is pure bullshit. Types *give* you control > they don't take it away. But you don't seem to know what a type is. > > Clifford Heath. >
I feel rejected. DonY hasn't written 20 reams obfuscating what he previously wrote about types. G*d forbid he's actually edumacating himself before responding? Or maybe he feel the need to write 2000 reams this time? :) Clifford Heath. (Don, I like you. I really do. You're passionate about the things that I'm sure you do well.)
On 28/06/17 21:56, bitrex wrote:
> On 06/28/2017 01:51 PM, Don Y wrote: > >> Exactly. You use the fetched "bytecode" as an index into a list of >> function pointers and invoke the function referenced. To it, you pass >> a pointer into the "instruction stream" so it can fetch "arguments" from >> the instruction stream, returning the updated "PC" for use by the next >> iteration of the interpreter. >> >>> You can have foolproof variant types that will happily hold both >>> function pointers and data in the same "package", on the same >>> software stack, using a custom stack allocator designed as >>> appropriate for the capabilities of the hardware. You can have >>> locally scoped lambda functions generated on-the-fly at compile time >>> and stored in flash memory that can be templated to accept any number >>> of arguments from the stack as is required automatically. >> >> "Typing" is something that requires careful thought. Often, you don't >> *need* >> a variety of types: a numeric type and a string type can cover a lot of >> applications. More types inevitably lead to more conversions between >> types, >> formatted printing, etc. My scripting language uses one of each and >> allows >> them to "grow", as dictated by the application. > > The C++ solutions available for discriminated unions on x86 platforms > are STL containers like boost::variant and boost::any. boost::variant > uses function pointers under the hood IIRC, while boost::any uses black > magic and is much more generic (and about a thousand times slower.) >
I don't know about the boost version, but C++17 has std::variant (and std::any). I think with compiler support for C++17, much of the "work" for these will disappear in the optimiser, at least for many use-cases.
> <http://www.boost.org/doc/libs/1_64_0/doc/html/variant.html> > > Functions have never been first-class objects in C++ but with C++11 and > some template library stuff they can come pretty close. You can pass > them around to other functions by reference or value, hold them in > containers, put them on stacks and in queues, whatever.
More importantly, you can create new functions within other functions and return them - that is key to functional programming. This comes from C++11 lambdas. (Yes, boost had lambdas before - but with C++11 they are part of the language and much more efficient, and marginally less ugly.)
> > I think numeric types should be "strong" and suited to the application, > basically classes. Stuff like "int" and "float" are really too broad. > Ideally it should never be possible to try to assign the return value of > something like a temperature sensor to another type that can hold say > complex numbers or floating point values way outside the range of > Earthlike temperatures - the compiler should complain. > > That is to say if one has to convert between different types with any > regularity it's probably indicative of bad design.
Agreed.
> >> The advantage, there, is that the user need not be concerned with things >> like overflow or ordering operations to maximize preserved precision, >> etc. >> >> The disadvantage is that you do a lot *for* them and take away a fair >> bit of control that they might want to exert on the performance of their >> code. >> >> [It also means you tend to need runtime GC -- which brings up another >> variable in performance (as well as more implementation decisions)] > > A pure functional language implemented via a stack machine doesn't need > a garbage collector, as there's no "garbage" to collect. Downside is you > do lose flexibility and performance. >
Pure functional programming languages /do/ have garbage collection as part of their implementation - if they want to run at a useful speed. In pure functional programming, a function "foo" cannot change the state of the world. Instead, you pass it the world as it is now as a parameter, and receive a new world as a return value. There is no garbage, nor any fixed data - everything is parameters and return values on the stack. In practice, of course, this is absurdly inefficient so the /implementation/ is decidedly non-functional in nature.
> My gimmick was to try to isolate the mutable state (i.e. when > interacting with external widgets through policies written in C++) > within APIs that aren't directly accessible as much as possible. > > Indeed I don't think a pure functional language that never had to take > any input or provide any output to the real world would require any > mutable state at all.
That's the theory - but not the practice (at least, not in the implementation).
On 6/28/2017 10:15 PM, Don Y wrote:

>>> I think numeric types should be "strong" and suited to the application, >>> basically classes. Stuff like "int" and "float" are really too broad. >>> Ideally it should never be possible to try to assign the return value of >>> something like a temperature sensor to another type that can hold say >>> complex numbers or floating point values way outside the range of Earthlike >>> temperatures - the compiler should complain. >> >> Again, it depends on the "market" you're trying to address. I have >> ONE numeric type and one string type. I don't require the user to >> set aside buffer space for strings -- there are no buffer overruns >> (until you run out of memory and the system ABENDS.) > > This is the sort of script a "user" might write (with the declarations elided > and a fair bit of hand-waving -- just to give a flavor of what a user is > likely to WANT to do and the amount of detail he DOESN'T NEED TO SEE). He'd > undoubtedly cobble it together from other scripts that *appeared* to work, > blissfully ignorant (an intentional consequence!) of all the machinations > happening behind the scenes:
[*my* messages sometimes take a while to make it back through my (personal) news server, so...] I think some of the elegance of this approach is too subtle, here. Think about how much detail is NOT exposed:
> while (count > 0) { > (process, error) := <= confirmation > if (error != nil) > print("OhMiGosh! " process " reported " error) > count-- > }
(remember, I'm writing a scripting language for *users*, not "programmers"; but, in which programmers can also develop applets) How, for example, would you write that "print()" function in another language? Is "process" a numeric or string type? If numeric, is it a float or an integer? How *big* (in reified form)? You can conceivably encounter "error/log messages" like: "OhMiGosh! 23 reported 45" "OhMiGosh! 23 reported motor overheated" "OhMiGosh! master bath reported 45" "OhMiGosh! living room window reported communication failure" "OhMiGosh! skylight reported 18.26" The user isn't bothered having to ensure a "format specifier" agrees with the types of the data being "print"ed. Does he really care what form an error_t takes? Or, a processID_t?? All he really wants is a way of mapping a particular "process" to a particular "window" and a way of differentiating a "memory error" from a "mechanism error"... A "software developer" might -- but would take greater pains to ensure things were printed as *he* thought they should be printed (e.g., KNOWING that he has specified error_t as a numeric type, he may want to display it in hex; or, that all processID_t's are strings and he'd like to line them up in nice, 23 character space padded, right justified columns; etc.) Late for my luncheon appointment <frown>
bitrex wrote:
> As someone pointed out a while back while the Arduino toolchain seems > to be very popular for developing hobbyist embedded applications for > AVR/ARM, but the API and IDE is kind of a mess. The API is a > bastardized subset of C++ where a lot of things don't work and which > most users only use to write kludgey, procedural C-like code; you > have C-like global functions such as "digitalWrite" which under the > hood use 40 or 50 assembly instructions just to write a bit to a GPIO > port. >
I think the constraint on project size means this is just fine. You don't want ctors()/dtors() nonsense buggering up a small micro. There is always assembly.
> I think it's certainly possible to make something better that allows > someone who doesn't want to fight with the tools get real work done. > > I was thinking that since little uPs are really more like > programmable hardware rather than general purpose computers, if one's > looking to make a microcontroller language for the everyman it > doesn't really make sense to derive it directly from a general > purpose systems programming language like C/C++. >
That's a really big wheel to reinvent.
> In modern computer games the graphics and physics engines are written > in high-performance compiled languages like C/C++. But the whole > product doesn't use those languages to define its behavior; usually > all the plot, mechanics, and other stuff that make a video game a > game is fleshed out in some kind of scripting language at a much > higher level of abstraction. This allows people other than rockstar > systems programmers to contribute to the design and avoids needing to > recompile the whole codebase every time someone decides this enemy > spaceship should fire purple lasers instead of green. >
A video game capable computer is many, many orders of magnitude away from an Arduino. It's skateboards vs. fully loaded freight trains. That's not even I don't know :)
> I think even 8 bit processors with modern architectures are fast > enough that one could adapt a similar paradigm to writing embedded > apps. The high-performance stuff, like interfacing with external > hardware through GPIO can be written in C/C++, in something like the > "policy-based design" paradigm where you have an abstract type of > device, like "Display" or "TemperatureSensor" which defines an > interface to the basic functions any device of that type should be > able to do, and then "policies" or "plug-ins" which handle the logic > requirements of some particular type of TemperatureSensor from some > manufacturer. If you want to change the sensor you don't rewrite the > entire codebase, you just rewrite the plug-in. >
This is much more trouble than it's worth. I'm working with someone right now who is taking Arduino class devices[1] seriously for instrumentation, and the point of an Ard is that it is nimble and deterministic. [1] as proto boards, with custom designs for production.
> And then the logic for how the external hardware interacts could be > written in a very abstract stack machine language, similar to Forth > or PostScript, or a programmable RPN calculator. An example stack > machine implementation for Arduino I saw would blink an LED on and > off like this: >
That's not a bad approach.
> 13 output { 13 digitalToggle 1000 delay true } while > > These scripts aren't compiled, but actually stored as plaintext > strings (in compressed mnemonic form) in Flash or EEPROM and then > interpreted on the fly by the stack machine.
I think you'd at least want bytecode. But ... https://playground.arduino.cc/CommonTopics/ForthOnArduino -- Les Cargill