[FoRK] PubSub NG Re: MQTT : Exploring the Protocols of IoT - News - SparkFun Electronics
Stephen D. Williams
sdw at lig.net
Fri Feb 27 18:53:32 PST 2015
On 2/27/15 5:08 PM, J. Andrew Rogers wrote:
>> On Feb 27, 2015, at 3:21 PM, Stephen D. Williams <sdw at lig.net> wrote:
>> I'm working on other things and haven't surveyed the state of this area recently, but given a lot of other background, off the top of my head:
>> Security: authentication, authorization, reliability (also see comments below)
>> Configuration: Messages, settings
>> Data: Bulk, periodic, streaming, subscription
>> Power: Timed intermittent, beaconing, low-power monitoring
>> Timing: synchronization, clock, time
>> Error: best effort vs. reliable, watchdog, recovery / restart, reliable firmware update w/ security and rollback
> These are not embedded platforms in the traditional sense but (usually) full-blown Linux servers at the edge in non-server form factors running sustained load. How the sensor data becomes available to Linux is not a significant concern; there is no reason for the transport to go all the way to the sensor. The correct mental model is a server as an endpoint, rarely smaller than 64-bit ARM.
I was considering the broad scale possible. It is close to being the case that you could assume an equivalent of an older Android
phone chipset at every endpoint, but not completely. Some situations are going to call for throwaway cheap as possible solutions,
perhaps in the pennies.
Your example below regarding large networks of high-bandwidth endpoints is interesting, and certainly should be handled. But there
may be vast numbers of a whole range of sensors, from a few bytes per second on up.
> Also, these sensor platforms are increasingly being built using the modern data center model: reliability through cheap, redundant units. If a unit fails, no one cares.
Of course, for the unit, but you still want data to get through reliably.
> The missing part is a protocol for a global-scale dynamic fabric that connects these “servers” together into local ad hoc computing environments given the constraints and typical applications for sensor data models and mobile platforms.
Agreed. And some ability to support models of ownership, leasing, temporary leasing, multipart simultaneous shared use, etc.
>> Security ought to be somehow done as an independent layer so that it can be replaced, fixed, done as a separate chip, etc.
>> Everything should be encrypted, signed, nonced, timestamped, and strongly protected under one or more security modes.
>> It ought to be possible to configure a device in simple shared key mode and alternately strong PKI/GPG modes. TLS (HTTPS, SSH, etc.) is probably going to be preferred for at least some modes.
> No need to over-engineer this, it isn’t the web. Thinking it is like the web is what is so broken about current protocols. No one needs HTTPS, SSH, GPG, etc for this. You can always proxy out to some other protocol at an endpoint if you wish or tunnel another protocol.
Isn't it like the web in some cases? If I want to access the entry-way camera, flood sensor, etc. in my building from anywhere, but
I should only get access if I'm verified as a resident?
> Assume you have SHA-256 and AES-128/256 in silicon, because you often do. Basic PK exchange is in software (RSA or maybe EC). That is the toolset.
>> Ideally, should support direct web-browser use or debugging, probably as an option, maybe via a standard mapping proxy.
> Wut? Do you realize that your PubSub stream in these models is 1-10 Gbps per source and that a single *logical* stream may be the aggregate of many sources? It is truly decentralized at a fundamental level because it has to be. You feed your client with a distributed constraint running remotely that is constantly moving between resources in the fabric such that it is never really running in a single, concrete place. Traditional PubSub applications have data rates that are sufficiently tiny that you can get away with a lot of subtle and not-so-subtle centralization in the protocol. Not so here.
> Indeed, what makes this different in the abstract is that centralization shortcuts do not work anymore. Can’t and doesn’t exist in big fabrics like this. Data is not located at a fixed point in the network and relativistic effects thwart a God’s eye view. Nonetheless, you can compose a locally consistent view of a logical stream using decentralized constraints within the limits of the local bandwidth.
> Trust me, you get used to it.
Interesting, but not the only use case. Nest thermostat, etc.
Filtering should be done at or near and endpoint when possible. However, that doesn't negate the concept of pubsub. I still may
only want a feed of events for a certain time, and as always, polling is bad.
More information about the FoRK