Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Thoughts about a post-3.1 world
So, it's become more and more clear that I've built a really great product for someone like me. Unfortunately, most installers out there are not software engineers (see, it's the details that get you.) Therefore, we've been thinking a lot about how to help on this front. Various ideas have been thrown around, but we are kind of honing in on an idea that we hope will help.

Nothing is going to REALLY make it simple. What we want to do is allow folks who are good at building solutions to do so and then be able to re-apply that solution (or more to the point allow others to re-deploy it) much more easily. You can currently build a nice solution, but it's a fair amount of work still for someone to reapply it if they have different devices and such.

And we also need something that will make the packaging up of an entire solution more feasible in such a way that it would be remotely useable when imported into another system. ANd of course a way for the meta-integrator like Vidabox to deploy improvements to their pre-fab solutions that are actually workable in a system that has been re-deployed to new hardware.

The idea if fairly simple overall. I just want to try to lay it out here at the 10,000 foot level for comment and criticism, to see if anyone can come up with any good reasons it won't work as planned.

Really, what it comes down to is that there's a lot of logic that needs to be created to create a good, solid solution, and that of course includes a lot of interface work. As long as that logic and those interfaces have to deal directly with the drivers, they will never truely be re-deployable to any great degree.

There are probably two fundamental approaches to dealing with this. One of course is to force all drivers to very strict standards. The problem with this approach is partly that the horse is already out of the barn. And partly in that it limits the power of the system for those folks who really are looking to build extremely powerful targeted solutions. The types of integrators looking to do that would likely be the higher end ones, but clearly we don't want to make the system less appealing to them.

Or, we could approach it from the other end and provide a way to create a virtualized view of the system, and allow you to write all your logic and draw all your interfaces in terms of that virtualized view. Then redeployment, for the most part though not always 100%, becomes a matter of mapping the virtual view of the system to a real system (i.e. to real device driver fields.)

The latter seems more reaslistic. It allows the system to remain as it is for those folks who want to work at the more open level. But it creates a means for integrators (and for meta-integrators like Vidabox) to create redeployable solutions.

So, the elements of this solution would be:

1. We add the abiltiy to set a 'semantic type' on any driver field. These would be at the level of Boolean Switch, Dimmer, analog sensor, digital sensor, volume, source selector, etc... This won't change any drivers, and it won't be required that drivers do this. So we can implement this feature in drivers as it's required over time.

2. We create an 'uber driver', which is really in two parts, not unlike the logic server in some ways. One part is a server and another part is a driver which treats that server like a device.

3. We provide a configuration utility which lets you configure that uber driver. That configuration will be composed of something like sites, floors within sites, and rooms within floors. In each room you can define faux fields, which will be of the same semantic types as mentioned in #1 above. So, if you want to add a light dimmer to a given room, you will get a list of all of the fields out there that indicate that they are a dimmer. That will then become a faux field in the uber driver.

4. Since the uber driver knows the semantics of these fields, it can automatically provide scaling of volumes to a standard percent range and things like, and it can also provide some manual control over ways that the faux field maps to the real field.

5. Once you have configured the uber driver, it then effectively has now created a virtual layer over your real drivers that hides the details. You can write logic in terms of that virtual view of the world. And, since it's a driver itself, any scheduled events, triggered events, etc... can work in terms of it, not in terms of the underlying real drivers.

6. I will add support to CQCServer to automatically forward field changes to the uber driver's server. So there will be no double latency charge for doing this. Changes will show up quickly. Field writes will of course become a two step process, but that's not a big deal. Those happen quickly. The uber driver's server will of course handle mapping the abstract view to the real thing, i.e. you write 50 to a volume field and it already knows from looking at the range limit on the real field how to map that to an actual value to write, and vice versa for readable data.

7. What would be even nicer is if the uber driver could automatically name all the faux fields to a standard format, based on site, floor, room, semantic type and instance number within that room (i.e. the 2nd dimmer in bedroom 1, on floor 2, of site 1.) This would allow for LOTS of auto-generation of logic and interfaces, since any tools like the interface designer can completely understand everything configured for a given room. I'm not sure if folks would buy into that or not. It would make for some kind of unwieldy field names in some ways, but the consistency and autogenerational compabilities would be very useful.

8. Since all of the logic and interfaces is written in terms of the faux fields of this uber driver, then all of the issues that we have now with trying to do search and replace through lots of complex logic to change this field to that field kind of go away. Changing to a new device is something that you'd do at the uber driver configuration level, where there is no problem. It's easy to find every faux light switch field and change it to reference a different real driver field.

9. Since the actual mappings to real devices is hidden from the actual solution logic, you can develop a system against just the device simulator drivers basically, and then redeployment to real hardware is a matter of remapping the virtual view to a new set of underlying real drivers.

So, anyway, that's it in a hopefully semi-coherent nutshell. Again, this would be a layer over the existing system, so it would be something you wouldn't really pay for at all if you don't use it, and it wouldn't impose anything on the existing system in terms of changes that you would see if you aren't using it. But it could provide a very useful means for creating reuseable logic and interfaces.

It obviously cannot 100% deal with some of the illogic of some devices, though even there we could do some things to make that more feasible. With such a system in place, we could then also get a lot smarter about being able to package up that entire virtual view of the system as a whole, instead of the much more piecemeal scheme we have now.
Dean Roddey
Explorans limites defectum
Place holder for some more commentary and examples....
Dean Roddey
Explorans limites defectum
I very much like the concept. I have been on a forward path of exploration in creating a system that is infinitely scalable yet fits within finite bounds. A bit of a dichotomy. I already have several activity based drivers that provide the abstraction between driver, interface, and the underlying hardware. Things like an audio controller driver that allow you to mix and match any number of audio sources and audio distribution devices to look like a single "Music" driver. In all cases naming of fields in the controller drivers is key.

Naming for ease of use in auto generation may or may not have its merits. However, having a consistent naming scheme will be paramount in making a driver like this usable. The size of the field space will likely be quite large and being able to move around in it efficiently is required. Being able to choose the granularity of the hierarchy would be good as well. Someone may just want a leaf type of tree where you just have rooms while someone else may want branches where you have floors, rooms, and so on.

Along with the standard naming scheme you use for the fields if you include the "descriptor" fields to go along with them it makes for developing transportable interfaces relatively easy. For example you might have a field that is RM01LL01 which would be Room1 LightLevel 1 lets say. Along with that you have a descriptor RM01LL01Desc or similar. For the room you would have RM01Desc. Things like that. Better yet would be to have a property of a field that is a descriptor similar to the semantic type. Probably better would be to have the faux field have a description property. Having that type of capability is what allows you to build user interfaces that transport easily from one system to the next.

It is possible that having one Uber driver may be a bit too much and in the desire to make things clearer some folks may still get lost in the fog. If a person could create a set of the abstract drivers that may be more palatable. An uber driver for lighting, one for audio, one for video, etc. That would make for plugging in functionality a little easier from system to system. Instead of having one system with xyz and one with wxyz you just build up a system out of the individual components.
It's the early bird that catches the worm, but it's the second mouse that gets the cheese...

Anything you can do to make deployment and re-deployment easier, would be good. But I think there may be some more system management type of things that would also be helpful. A while back I put together this list. Some of them may have been done already. Here's my original list:

1. The ability to export and import events. I have a bunch of small events in my Pluckers configuration and if I edited them on my development system and then needed to install them on one of the production systems, it would be a real pain. A batch export and import would also be nice.

2. Could a date code be added, or may already be there, to templates? Then in the Admin I/F, show a field that is the date saved or something like that.

3. I’m having difficulty getting Pluckers to agree to and pay for a VPN into the CQC master servers in their stores. But they do have email setup on the systems. Would it be possible to have a feature in CQC to install an outgoing mail service. This service would be able to email logs, driver states, errors, performance issues, … to a particular email address. This way the IP could at least monitor their systems without actually logging into them.

4. Packaging up all the templates for a new system in one operation instead of 79 (in my case). I think you are creating a folder - subfolder scheme for templates. Maybe there's away to, with one operation, package all the templates in a subfolder.

5. Importing the templates to a new system, could that be one operation?

6. Installing drivers. It's quite time consuming to install 50 copies of the same driver. Maybe that operation could be command file driven. Something like putting the driver name, moniker and com port in a batch file.

7. Get driver statistics out of all the drivers. Log results. Email log/results.
I got really excited when I started reading your message. The idea of a virtual development area as it were, is something I think would help everyone, integrators and non-integrators. Connect to a system from an self-contained environment, suck out its config and allow changes to the system before putting them back in place. A standalone development environment as it were. This would need to include a way to emulate drivers and values (like your simulator driver) but all from inside that one little package. I've often wanted to be able to connect from my laptop, work on the system but really I don't have the patience to do the tweaking of an interface via RDP or the like. This would meet the deployment requirements of the integrator as they can push a configuration to multiple places.

If you do decide to head that route, the integrators probably would love a change-control like apparatus. I know I've done things that just didn't turn out right then regretted that and had to reverse the changes manually.

In the downloading and uploading the config, it probably should deal with Bryan's request #3 by allowing the creation of a file that can be attached to an email. Heck, it'd be really cool if there was a driver or service of CQC that could monitor a POP *AND* IMAP mailbox to handle remote configuration and monitoring. Built the right way, someone would definitely go the additional step of doing a front end to quick view clients etc.

OK - so this is my placeholder. This deserves a day or so of thought. No quick answers here. I will get back to this in two to three days.

By the way Dean - Great Post. :cool
Dave Bruner
My only concern with what you propose is the size of the task and the time it would take to figure out (if possible) the complete virtual model. I think it is probably harder than it appears on the surface.

I wonder if there isn't an incremental step or two that could be taken in a point release to make templates more portable that would result in a lot of usefulness in the short term.

First, as mentioned above, there is a real need for import / export of the event system.

Second, I could imagine a shorter term step that would be template pack centric that would very useful. You could create a new section of a template pack (collection of templates), that would have information on the drivers and events used in the collection. This seems like it could be automatically generated on export with some possible user interaction to add detail that isn't extractable mechanically.

On import, the system would prompt for how to map the template driver references to the system. This could be implemented as some sort of translation table for the template to convert template driver name to system driver name. Variable driver accesses could optionally auto-create the required fields and even add the variable driver to the system if not already in place. The system would auto-create the event triggers on the required drivers based on event usage info collected during the export.

Maybe with some tweaking, this could also be a step along the way to your virtualization idea.
The virtualization stuff itself doesn't necessarily have to spring full wrought from the mind of Illuvatar either. There will probably be a few plateaus to hit, each of which is quite useful and well worth doing, if not the final answer. But, given how key it will be, I kind of want to start on that, so that we have the maximum amount of time for it to be evaluated and refined for the next release.
Dean Roddey
Explorans limites defectum
Documentation in a Wiki!
For the normalization of field values for things like volume and such to a 0-100 scale are you planning on using look up tables or straight forward y=mx+b type stuff? You will find when interpolating using the linear transformation that values going in and coming out don't match up at times so you will get "flutters" in values. Although from your point of view you will probably just be looking at field changes so it won't be an issue really. The only thing that could be noticeable is when setting something from the actual device vs an interface something that should give the exact same value may be off by a tick.
It's the early bird that catches the worm, but it's the second mouse that gets the cheese...

Possibly Related Threads...
Thread Author Replies Views Last Post
  Some early thoughts on 5.4 Dean Roddey 40 5,031 07-20-2018, 01:37 AM
Last Post: Deane Johnson
  Post 3.1 Quirks/Wish List jrlewis 162 29,458 09-20-2010, 04:03 AM
Last Post: batwater
  RIVA Web Image Widget support thoughts/questions for post-3.1 SamVimes2 3 2,429 02-01-2010, 06:33 AM
Last Post: wuench
  A post-3.0 idea to work out... Dean Roddey 51 14,563 07-06-2009, 03:08 PM
Last Post: Dean Roddey
  Official Post-2.0.14 beta thread Dean Roddey 110 25,585 03-17-2007, 10:16 AM
Last Post: Dean Roddey
  Post 2.0 Feature Requests zaccari 174 32,244 02-25-2007, 05:51 PM
Last Post: Dean Roddey

Forum Jump:

Users browsing this thread: 1 Guest(s)