OaF – Oaf Aggregation Framework

The OaF was originally inspired by the Ambient Orb, a nice little pager network driven indicator which gave the promise of telling you how things are going. It seems to me that it’s most successful application was in the power industry where companies issued them to consumers to help guide their power conservation efforts.

When I saw this in the early 2000s I had a different take on it. My thought was that I wanted one indicator that just said “everything is ok” maybe with a few other states like “your process is running,” “warning,” or “error.”  This required two features first a hierarchy of aggregated statuses so warnings could be overlapped and managed in terms of importance and second powerful networking support which I got from twisted.
Implementation wise OaF has support for inputs from web forms, xmlrpc, email monitoring, and http ping. Outputs include the original Ambient Orb, the system tray, Second Life objects, Arduino on the serial port, and a web page. At this point I’m releasing as is and will work on making sure all this works on the next pass.

1. Grab the code from SVN and post it into GitHub:  DONE https://github.com/Ciemaar/oaf
2. Write up an introductory blog post:  DONE, this post here.
3. Make the code work:  IN PROGRESS, as posted it sort of works.
4. Attach the code to Codacy to run analysis:  DELAYED, Codacy is asking for rights to all my repositories and I can’t give them.
5. Build a deployment package of some sort:  IN PROGRESS, setup.py is included, but not fully tested.
6. Put up another post here with more details and an open source license:  NOT STARTED

Long Delayed Releases

I’ve decided that it’s time, well past time to get a bunch of my old releases so, over the next few weeks I’m going to be going through some of my old code and just getting it out there.  I’m also going to be grabbing my old projects–even the unfinished and bad ideas–from SVN and dumping them into GitHub.  Here’s the process I plan to follow:

1. Grab the code from SVN and post it into GitHub.  I’m not going to keep any history here, just check to make sure I’ve got the passwords out.  Moving passwords to config files is going to be the main part of my work.  At this point I won’t apply a proper open source license, but I’ll be glad to take requests if anyone sees a use for the code right away.
2. Write up an introductory blog post here explaining what the code is supposed to be doing and why.
3. Make the code work, in many cases this is very old code so it’ll have to account for new versions of twisted, Python, and webservices.
4. Attach the code to Codacy to run analysis.
5. Build a deployment package of some sort, probably pushing code to pypi and/or dockerhub.
6. Put up another post here with more details and an open source license.

So, the next few posts on this blog should follow in this line–at least as much as I can over an otherwise plenty busy holiday.

Automation on a Rainy Day

On my most recent flight back from Toronto I got to witness a modern airline functioning with none of their automated systems.  I was flying the generally excellent Porter Airlines when one of their vendors apparently did a mid-morning patch.  Of course it was not supposed to impact anything, but it did.

Continued on the Risk Focus Cloud Solutions Blog

Handling Types in Python – Enforcing Types with Properties

Building on my thoughts from Handling Types in Python – Ignore it, Convert it, or Check it, I’m continuing to build up to my talk at PyCaribbean in Santa Domingo Dominican Republic. My title will be “Python: As much typing as you want,” obviously this means that I’ve been doing some thinking quite a bit about Python typing of late.

Previously I looked at the three basic strategies Python has for dealing with types; ignoring them, converting them, or checking types. All of these are very practical in their own right, but basically limited to working inside a given function call. However Python is an object oriented language and we do have the option to work with that and create an object attribute which has a type enforced on it.

Getters and Setters

Getters and setters are a common way to handle this. We could write these in Python, it would look something like this:

def get_age(self):

    return self.__age

def set_age(self, age):

    self.__age = int(age)

norm.set_age('23')

assert norm.get_age() == 23

Of course this is combining with a converting strategy to ensure that age is always an integer, set_age(‘Green’) will cause a ValueError, obviously we could do type check instead.

Properties via @property

Of course in Python we don’t do it that way. In Python we can create a property using the @property decorator like this:

@property

def name(self):

    return self.__dict__['name']

@name.setter

def name(self, value):

    self.__dict__['name'] = str(value)

norm.name = "Andy"

assert norm.name == "Andy"

norm.name = 42

assert norm.name == "42"

Again we can enforce any amount of typecasting either within the getter or the setter

Properties via property()

Since decorators were added to Python in 2.4 this has been by far the preferred way to declare properties. However, before decorators we already had properties the syntax we used then is still usable and for enforcing types it’s really handy. Combining our getter and setter from the age with the alternate syntax we can create an age property:

age = property(get_age,set_age)

norm.age = '28'

assert norm.age == 28

in and of itself the age property is like any other. This will however allow us to get some much better code reuse. Let’s assume we want to have a few properties with the same constraint, we’ll enforce them as Decimals so we can store money. Let’s write a function that creates a set of getter, setter, and deleter for this purpose:

def create_gsd_convert_Decimal(name):

   def getter(self):

       return self.__dict__[name]

   def setter(self, value):

       self.__dict__[name] = Decimal(value)

   def deleter(self):

       del self.__dict__[name]

   return getter, setter, deleter

now we can use it like this:

price = property(*create_gsd_convert_Decimal('price'))

price = 380.23

norm.price = price

assert isinstance(price, float)

assert not isinstance(price, Decimal)

assert isinstance(norm.price, Decimal)

assert not isinstance(norm.price, float)

using a function to return a tuple is a bit clunky, so lets clean that up a touch, I’ll continue to use the function creator function above adding only the call to property:

def enforce_Decimal(name):

   return property(*create_gsd_convert_Decimal(name))

much cleaner to use now

tax = enforce_Decimal('tax')

The interface here is pretty much as clean as it’s going to get without introducing a metaclass, but in my talk and a future blog post I’m going to explore how to use the descriptor protocol to simplify the mechanics behind this.

Handling Types in Python – Ignore it, Convert it, or Check it

Next month I’ll be speaking about Python types at PyCaribbean in Santa Domingo Dominican Republic. My title will be “Python: As much typing as you want,” obviously this means that I’ve been doing some thinking quite a bit about Python typing of late.

Sometimes we think of Python and other scripting languages as devoid of types. Of course this is not true, almost every language since Assembly has included types to a greater or lesser extent. Even features like Perl’s coercion are just built on top of and intended to convert types—not based on ignoring types.

In the case of Python all of our variables are actually a single type, reference. They are all references to objects and those objects are typed. You can assign any reference to point to any object, but those objects have a class, which is their type and controls their behavior. You can see Python’s types at work if you do something like 2 + ‘2’ which of course equals TypeError. We can check the types with type(3j) or isinstance(42,(int,float)). But how do we deal with these types in Python? Python has 3 common strategies.

Ignore it

One very common strategy in Python is to leverage the fact that all of our variables are references, your code may not care what it was fed. If your code doesn’t do anything where the type matters maybe it doesn’t care. For example:

print(self.x)
print(self.right > self.left)
return pickle.dumps(self)

In all cases here it doesn’t matter what the type is: it will be printable, it will be comparable, it will be possible to turn it into a pickle.

Convert it

Our next strategy is to convert to the type we want, just to be sure that the type works as we need it to. This also has a few variations:

self.x = int(self.x)
self.array = list(seq_or_list)
self.value = value*1.0
return self.x * self.array

These techniques are often working on the convention that python constructors return a new object of the same type, making them safe to run on an object that’s already the right type.

Check it

A less favored strategy in Python is to check the type of data explicitly. We can of course do this, but this is by far not the norm. As soon as we start doing type checks—no matter how obvious the need—we break the Python norm of duck typing. Nonetheless there are a few cases where we do exactly this.

assert isinstance(self.x, str)
if not isinstance(self.left, self.right.__class__):
    raise TypeError('Left and right must be matching types')
if not all(isinstance(x, (str, int, float, bool, None)) for x in vals):
    raise TypeError('Unable to convert to json')
return 'OK'

Python is even doing this itself. Consider what happens in Python 3 when we compare two different types.

>>> 1 > ‘2’

Traceback (most recent call last):

File “<stdin>”, line 1, in <module>

TypeError: unorderable types: int() > str()

What else?

There is another category of techniques, more common to developers in other languages. We can convert or check the types when they are set as a field on an object. In other languages this would be getters and setters, but I’ll explore the Python equivalent in another post.

The prototype for my Raspberry Python:Call Mom Button class this Sunday @ MakerBar

I finally got time to hack together the prototype for the class project for my class this weekend. We’re going to be using the RaspberryPi to connect to Plivo and make phone calls, nominally to our Mom’s. I thought one button was boring so our call Mom Button class will actually have 4 buttons suggested as:

* Call Mom
* Call Dad
* Text Mom “I love you”
* Mother’s day special

For more details or to sign up see Meetup.

Cross posted on MakerBar Blog.

Not Making Money Ain’t Cheap

Last January one of my friends, Bert Hartmann, approached me saying it was that it was time to finally found a Hackerspace in Hoboken.  He’d already gathered a half dozen people and thought I should be in on it.  Having been a long time admirer of hackerspaces without ever having one close enough to join I was in.

We met through the spring looking for how to do it, what to call it, where to put it, how long to wait,and how to do it legally..  There were a lot of options, the easiest of which would be to become a for profit company, that just doesn’t make any money–that would be really easy.  We are not doing that, instead a year later and with the MakerBar well established we are filing to become a true, legally recognized,  charitable educational organization. This will cost us about a thousand dollars in filing fees, but it’s the right thing to do legally and simplifies our future taxes and donations.  Still this is a thousand dollars we’d rather spend on new kits, power tools, publicity, and other things that support our mission and what’s worse it’s a thousand dollars we need now–before the taxes are due.

So we’re having a fundraiser.  We’ve teamed up with The Melting Pot in Hoboken to have a charity auction and happy hour.  We’ll be selling some projects that we’ve treasured through the year and some that we build specially for this event, sharing our love of hacking with the community, and enjoying a chocolate fondues contributed by the Melting Pot.

More details are on the the MakerBar Blog here and tickets are for sale on our Meetup here.

Raspberry Python – Talking Clock @ Hoboken MakerBar February 9

Next Saturday I’ll be teaching another class at my hackerspace, the MakerBar in Hoboken.  This will be my third Raspberry Python class.  Following up on the initial class, and the train class.  Like the others, this workshop will cover the basic use of a Raspberry Pi, but it’s coming together really well, I’ve got a Pi sitting on my desk chattering inane things from my twitter feed at me and actually, it’s proving more useful than I expected.  Just in developing the project I’m finding more out about my friends than I had before–mostly some corporate feeds to unfriend and some friends to check up on.

The class will run about 2-3 hours so we’ll have time to get our Pis setup, on the latest version of Raspbian, which already includes NTP, and really dive into the programming and improvement.  Since most of our time and effort will be software, the hardware is perhaps not so impressive, no loop of train track here, but this is a picture, of the prototype

If you’re interested in coming, please rsvp at meetup here.  And if you need a Pi or SD card, leave a comment there, a few are on order so we should have some.

The First Clock of February

By the end of the week it will be February and my hackerspace, MakerBar, will be on to our February theme, it’s about time. The members are talking about clock and calendar projects and there are two clock classes in the works; Raspberry Python – Talking Clock Feb 9th and ChipKIT for Organic Lifeforms: Clock Edition Feb 23th.

This then is the first clock of February, a ChipKIT Uno32 with a ChipKIT Basic I/O Shield attached. As befits a beginning this is a very simple project.

Step 1 Solder in RTC Crystal (based on instructions here)

Step 2 Attached Basic I/O Shield (got this for contest here)

Step 3 Program (combined OLED example here with RTC example here)


#include <IOShieldOled.h>
#include <RTCC.h>

void setup(){
IOShieldOled.begin();
Serial.begin(9600);

// Initialize the RTCC module
RTCC.begin();

// Set the time to something sensible
RTCC.hours(11);
RTCC.minutes(45);
RTCC.seconds(0);
RTCC.year(13);
RTCC.month(01);
RTCC.day(29);
}

void loop(){
char date[9];
char time[9];

// Format the time and print it.
sprintf(date,”%02d/%02d/%02d”, RTCC.day(), RTCC.month(), RTCC.year());
sprintf(time,”%02d:%02d:%02d”, RTCC.hours(), RTCC.minutes(), RTCC.seconds());

//Clear the virtual buffer
IOShieldOled.clearBuffer();

//Chosing Fill pattern 0
IOShieldOled.setFillPattern(IOShieldOled.getStdPattern(0));
//Turn automatic updating off
IOShieldOled.setCharUpdate(0);
IOShieldOled.clearBuffer();
IOShieldOled.setCursor(0, 0);
IOShieldOled.putString(date);
IOShieldOled.setCursor(0, 1);
IOShieldOled.putString(time);
IOShieldOled.setCursor(0, 2);
IOShieldOled.putString(“MakerBar”);
IOShieldOled.putString(“ChipKit Clock”);
IOShieldOled.updateDisplay();
}

This clock will become the basis of the class project for the February 23th class,  ChipKIT for Organic Lifeforms: Clock Edition. The clock in that class will not rely on this shield and will have functionality this clock does not, like setting the time, and time permitting an alarm.

Full, latest code available here.

PythonAnywhere

There is a missing middle in our approaches to sharing servers.  On one end we have static web hosting where all users are doing the same thing so we can install and run one copy of Apache for all users, that works up to the point of cgi scripts and php. The other end of the spectrum is to buy a server and that extends down to virtualized servers. Between there is a gap where there is some commonality to the user’s needs, but little support. Engine Yard has their RailsEngine platform for Ruby and there are a few Python offerings of which PythonAnywhere is one.

PythonAnywhere starts from the assumption that you will be running Python. They also assume that you’ll want to use Python interactively at least sometimes, probably a lot. The operation of PythonAnywhere is to login and select a console. You can choose from your existing consoles or start a new one in your choice of:

* Python: 2.7 / 2.6 / 3.2
* IPython (0.12): 2.7 / 2.6 / 3.2
* PyPy (1.6): 2.7
* Bash
* MySQL

Once you select a console it will open directly in your browser through a combination of JavaScript and Flash. It tries a few different connection mechanisms so it does work even without Flash and through some but not all firewalls. It also works on Android 4 and supposedly iOS, though you should get a keyboard without auto-complete or an external keyboard . This kind of quick access to different runtimes is great for testing, but the consoles are also persistent, so you can start working on one device and switch to another.  You can prepare a session on your desktop and then connect to it from your laptop for a presentation, or rely on iPython’s history feature to recover even if it gets restarted. The other payoff is sharing, any console can be shared–no account needed for the recipient. Sharing is real time and persistent a class can share the teacher’s console passing control from student to student to answer or ask questions.

Surrounding the console features PythonAnywhere provides file hosting with a basic JavaScript IDE which will let you run scripts or launch them to a console. If you use Dropbox you can set it to sync folders or use your choice of version control to check out projects.

For non-interactive hosting you can go simple with scheduled tasks or hook in a wsgi application.  Because the servers are virtualized via Amazon’s ec2 you cannot run a server directly.  Free users can have username.pythonanywhere.com addresses, and paying customers can point their own domains at the service.  That’s pretty much the way of things when it comes to paying, PythonAnywhere is on a freemium model, most of what you need is in the free tier, unless you’re doing something that you’ll have to admit is on the heavy side.  The only real hassle to the free tier is that they’ve had to restrict http access to a whitelist of sites to prevent misuse.

To get started with PythonAnywhere just sign up at https://www.pythonanywhere.com/ with the free account you can get two consoles, a database, and everything you need.  Sign up is faster than installing python locally.  Even from the basic account you can share a console, so this makes a great tool for learning and exploration.  Give it a try next time you’re experimenting or collaborating, try it on a desktop or laptop before trying to use it on a mobile device where it can be a bit more finicky.

Disclaimer, I have been involved in the PythonAnywhere Beta test and they have thanked me with account upgrades and free gifts.  No one from PythonAnywhere nor Resolver Systems was contacted to prepare this review.