cratuki.blogspot.com

Response to Kelly, "Recession we don't have to have"

2008-10-14T12:01:00.000-07:00

"Recession we don't have to have"
http://www.theaustralian.news.com.au/story/0,25197,24497984-12250,00.html

It is comic that the Australian government has our bubble housing sector [1] as a place in which to focus yet more capital. This was a sector that the US propped up in 2001, with the subprime disaster being the result. The US should instead have allowed a natural correction, as should we now.

People need a natural balance of risk and reward to act rationally. As a result, economies have natural cycles where people who have overextended themselves are brought back to reality. The notion that the government can change this is delusional. All governments can do is to delay the pain. In the meantime, interventionist governments reduce the incentive to genuinely create wealth and savings with taxes and inflation. This hurts us right now.

From the way you write, anyone would think the government itself created value and was doing us a favour for finding new excuses to redistribute our money. They are merely defering the pain to future taxpayers. That situation is particularly tragic in the case of the United States. The astronomical debts that the current generation of Americans have taken on makes their children slaves to the nation that their ancestors founded in the name of freedom.

[1] http://news.google.com/news/url?sa=t&ct=us/0-0&fp=48f4042833fecfcd&ei=7uz0SLD0C5mWQ46a5JMN&url=http%3A//www.abc.net.au/news/stories/2008/10/14/2390735.htm&cid=1257693335&usg=AFQjCNHp78sI4lpUT2TWtzGlWR1YtktvJw

Error - printing

2008-08-13T00:23:00.000-07:00

When printers break you get a throwback to the dark ages of computing where nothing gave useful error reports and apparently simple things sit there and hang for minutes at a time. I'm waiting on a printer to delete at the moment. "Deleting", it says.

Has anyone ever seen a printer with a command prompt that you can be accessed via telnet?

Steamdoc

2008-07-10T11:14:00.000-07:00

Still pining for my long-lost Synect publishing system, I've recently knocked together a quick engine I can use in its place. It's called steamdoc, it's nowhere near finished, but it is usable. See the notes at songseed.org for details.

Python scoping

2008-06-05T10:37:00.000-07:00

I love python, but am annoyed by some of the quirks of its scoping. These are the sorts of situation that set me back to work on my project to implement a scheme notation for the python vm. That seems a bit extreme though because with these issues aside I'm quite happy with python as it is and figure that there's not a whole lot of distinction between using parens to structure a tree and whitespace. I even saw a proposal from someone to implement support for whitespace structuring to arclanguage.

Suggestions for my problems are most welcome.

1. Can't directly bind functions to anything but the current scope.

In python there are three ways to declare a function that I know of and one of them (eval) is sufficiently disgusting to be off the table. The two sane mechanisms are (1) the 'def' syntax which causes a function to be injected into the current scope and (2) the 'lambda' syntax which can't exist over multiple lines, which must return, and where the only way to get side-effects (like output) is via impractical hacks.

Here's something I do quite often:


def process_user_input(cmd):
    d_cmd = {}

    def cmd_load():
        print 'stub for load functionality'
    d_cmd['load'] =  cmd_load

    def cmd_save():
        print 'stub for save functionality'
    d_cmd['save'] =  cmd_save

    def cmd_quit():
        sys.exit(0)
    d_cmd['quit'] =  cmd_quit

    if cmd not in d_cmd.keys():
        print "Command '%s' is unknown."%cmd
    else:
        d_cmd[cmd]()

That is much nicer than having to declare each function and then have a laborious 'if' statement for each case as well. But this could be much better. Imagine this instead:


def process_user_input(cmd):
    d_cmd = {}

    d_cmd['load'] = fn(): # 'fn' is a fictional keyword for demonstration
        print 'stub for load functionality'
    d_cmd['save'] = fn():
        print 'stub for save functionality'
    d_cmd['quit'] = fn():
        print ".. guys ... home .."
        sys.exit(0)

    if cmd not in d_cmd.keys():
        print "Command '%s' is unknown."%cmd
    else:
        d_cmd[cmd]()

There's far less room for copy and paste errors in this. But it's not supported. It's possible you could inspect the scope of process_user_input to find out if the function is available there. But that's a lot less pleasant than what I'm writing above. I frequently wonder why it is that the python team prefer to have programmers do either laborious 'if/else' clauses or else
duplicate strings as I've done above (think a copy / paste error in the first example where you were duplicating a function).

Perhaps an decorator mechanism could allow the functionality to be improved. But that would still be inferior to the 'fn' keyword I mention above. Adds an extra line and requires the annotation to be declared somewhere. Further, I for one find annotations much more complicated to understand than the idea of a function being directly inserted into a dictionary.

Ooh :) Decorators are definitely not bad. Thanks to #python and quotemstr in particular for this snippet:


def dict_add(d, key):
    def _deco(func):
        d[key] = func
    return _deco

test_dict = {}

@dict_add(test_dict, 'a')
def _():
    return 'sup'

print test_dict['a']()

2. Rebinding variables

While I was writing about a different scope problem I thought I'd write this.

It's not possible to rebind a variable that's held in an enclosing scope, with the exception of its treatment of the global scope. This is stupid, because it just acknowledges that what I'm trying to do is legitimate but provides a disgusting mechanism for achieving it.


def reformatter(debug=False):
    x = 0
    def inc():
        if (debug):
            print "x is being incremented"
        x = x+1
    def dec():
        if (debug):
            print "x is being decremented"
        x = x+1
    # ...

This would work:


def reformatter(debug=False):
    x = [0]
    def inc():
        if (debug):
            print "x is being incremented"
        x[0] = x[0]+1
    def dec():
        if (debug):
            print "x is being decremented"
        x[0] = x[0]+1
    # ...

So that's naturally what I end up doing when in this situation. Devteam are doing something about this in python 3k. But not in way that gets rid of global variables as I'd choose.

3. Classes - something python gets right

The annoyances I've listed also get to me in java. But python's handling of
classes pleases me.

I've found this very useful in writing ORM tools.


def entity_person():
    class Cl(entity):
        def __init__(self):
            pass
    return Cl()

Still, I think there's a lot to be said for the approach used by languages like io and javascript where there's no distinction between objects and classes, and where you can thus easily modify the functionality of a class at a later date. Io is particularly sexy in that it has methods to allow you to dynamically retrieve the syntax trees of its functions, and redesign them on the fly if you like.

Process for programming

2008-05-05T11:27:00.000-07:00

"An Under-Appreciated Fact: We Don't Know How We Program" (link)

This is very interesting. I'm going to comment on some of his points, and then discuss my process.

"Like many developers I took a 3 year degree course in this
stuff. But at no point during those three years did any
lecturer actually tell me how to program."

This was my experience as well. There is a particular skill to finding libraries that cover functionality that you need without undesirable side-effects and finding ways to immerse them elegantly into your product. This is the major technical skill required in the sort of work I do these days and this is far from unusual. I say "technical skill", but there is a degree of mysticism about it because you are constantly on the lookout for warning signs that point to something that may trip you up in the future. As far as I remember we also got zero guidance on this at university. Fair enough - it's not necessarily a "computer science" issue. But the author's original point is quite valid - lots of people get into work without vocational training and the industry doesn't supply it.

"The problem for software is that non-software-developers
don't see anything creative about it."

There are sectors of software development that are not remotely creative. I remember working on a contract for some software used by a government department. It was written using a long-obsolete visual basic platform. My task was to churn along using the same approaches that had always been used on the project. We had complete confidence that the client would pay for this. There was no business case to justify extra money to improve it in the short or medium terms. If it ever became necessary it could have been funded in the future.

Of course, I agree with the direction of quote. This sort of work exists, but only because there are programmers apathetic or naive enough to continue working on such stuff. There's no way I would do it - it's boring to the point of suffocation and a career death-sentence. I'd rather work the front of a pizza shop and this nearly eventuated.

Now for the general comment.

In my natural state I'm not at all methodical even when I want to be. I need a single point of entry. As a result, I have processes to manage working on all things that are important to me at work and in my private life. For example - I have a paper diary; a small black notebook that I write a short list of coming goals in before I allow myself to unlock my workstation; I have a document on the computer listing current projects and milestones. And then whenever I get uncomfortable I can force myself back through the system from a single point of entry. I expect that I spend a lot of time managing finicky documents that other people are able to instead direct to actual productive things. But that doesn't matter, because I need this in order to function.

Thus, I'm probably unusual in that I do actually have a conscious process for programming projects.

This is it (elaboration added for purpose of blog post):
- Establish the business requirements as best as possible.
- Get an authority to agree that it's a good direction even if not final.
- Breadth-based brainstorm of technical approaches to properly meet business requirements. (Don't stop at the first one that looks like a good idea - keep searching. Discard "that almost works" type solutions - if it doesn't work it's broken. Doing this stuff on paper now saves lots of time later.)
- Put the options in a conscious hierarchy of how you find them flexible. (I don't really have a strict rule for doing this yet, but more a list of things like "future flexibility", "maintainability", "technical risk")
- Use "bottom-up" approaches to create demonstrations of each point along the path of an approach. Work towards building a prototype from the various components.
- Seek review on prototype. Expect the requirements to change. This is fine - people often need a prototype in order to move their understanding of the problem along. Also, better now than later, and there's a good change you'll have use for the scratch prototypes later.
- Go back to breadth-based brainstorm until the prototype works and the target audience are satisfied that things are headed in the right direction.
- Agree with authority that prototyping will now be over. Write a design document. Get signoff. There's still lots of work to be done - build systems, developer documentation, and you want to be secure in this before you jump in. (Sometimes you just can't get your audience to accept the shift from prototype mode to endgame. Such is life.)
- Implement.

Throughout you insist that people respect the pigs and chickens dynamic and give those who are actually and in fact committed to the project the ability to call it. This is an interesting one. Sometimes someone who is managing seven projects can say "well I'm responsible because I'm your manager". It's a lie. In three years time, everyone who matters will have forgotten how the power structures were arranged at the time and all that will be remembered is "This was Craig's project and it's rubbish". I've even been in situations where the absent minded manager who insisted on an approach forgot about his insistence and blamed me for the bad technology decision. Nightmare.

Random: I feel constantly hampered by not having a document production system at my disposal that gives me the power I need in many of these steps, and others besides. I sit down every now and then and have another shot at working out exactly what I need and implementing it.

Blogs as an email replacement mechanism

2008-04-13T08:19:00.001-07:00

This is written under tight time constraints and I realise I still haven't dealt with feedback from the previous post yet.

I've been thinking recently how neat the twitter model is whereby it avoids spam, and I've also been installing a new mailserver today. Hence the following notion.

I wonder if it would be practical for people to replace a large amount of their email activity with weblogs. For example - I could keep in touch with my family via a private-access "Turner family blog" that people in the extended family posted to when they wanted to communicate to anyone. There's nothing revolutionary in this idea, but the idea of actively trying to push my activities to private network blogs in order to cut out email is something I hadn't considered before. Thoughts?

Blogging has become more a log more significant to me since the beginning of this year. I'm increasingly using weblogs as a documentation and live communication system. For example - I've just set up a new blog for the church group I sing with on Sundays with an intention that we'll use it for program and event announcements. I'm in the process of setting up another for tracking my internal documentation for songseed.

I like being able to use a single tool (in my case, MarsEdit) to edit personal documentation, edit work documentation, dispatch python geek blog posts to one blog, dispatch choir geek posts to two other blogs. A generic "network editor". A browser isn't such a good interface for that work because there's always the risk that something will crash and you'll lose all your data. I suppose there's the same sort of risk with MarsEdit, but so far it has been completely stable for me, and it's more feasibly to put redundancy measures in a single purpose tool like a "network-based text editor" than to expect both a remote webserver and local browser to co-ordinate data safety.

Continuing the thought: if we used blogs extensively in this way (when you get a new member to a community you just invite their email to the community blog) then we could go further in securing private email agaginst junk. For example - it would be more reasonable to filter people through capcha in a world where the only time you were getting personal email was from someone you hadn't met.

Another thought: maybe we could use a blog approach where we currently use technical forums. So people interested in a topic pass through capcha to join the posting group, and anyone can post. Threads develop as comments to blog posts rather rather than having the back and forth you see on an email list.

C

"Push" technology is so 1998

2008-04-11T14:57:00.001-07:00

Back in the late nineties there was a fad called "push". Pre-Internet media is threatened by the pattern of consumers being in charge. "Push" was an attempt to redefine the emerging online space to something they were comfortable with. The vision was that people would connect to the net each morning, have their computer download the sort of content they were usually interested in and then they'd consume it.

I was an administrator at ninemsn at the time so had extra awareness of all this. They tried all pushing all sorts of things. They were quite innovative but nothing crossed my path that ever won me over to the idea. Time has shown that people seem to prefer news aggregators.

More recently, "podcasting" emerged as a fashionable word. This struck me as strange because to me it essentially the same dynamic as push content, except it's audio files instead of Dolly horoscopes. But the twitter about podcasting didn't die down, so then I started to get insecure and think that I had missed something. I've just read an article showing that experience has matched my expectations: see here.

There's definitely a place for podcast-style content, just as there is a place for mailing lists that you subscribe to. I like to listen to Graham Abbott's Keys to Music show (and wish I could get it in a more convenient format than is currently available to allow me to cart it around - I expect the current format is a result of copyright limitations on the content). This is the sort of thing I'd get to listen to in my car, and is superior to something like radio, because I get to pick out content I want and listen to that rather than listening to what's being broadcast on a small number of channels that are aimed at a general audience.

Still, the sweet spot for content is where it is driven by people's immediate active interest. As an example - imagine a world where songseed actually had a whole stack of content in it, and a consumer could view a composition and then see a "Keys to music" episode that related to that content. This would be a superior delivery of that content to a system where a loyal audience keeps the faith by listening out each week with a sense of confidence that the thing to be broadcast will match their interest.

Another thing - the fact that the twitter took ages to die down reminds me of another thing I've been thinking recently: popular wisdom about the rate of change is garbage. There's a common opinion that computers are obsolete before they're released, that this morning's newspaper is old news and that progress in technology is generally extremely rapid. My experience suggests that this is complete and utter rubbish. I sat around a geek computer meeting in 1995 discussing mp3s. Everyone there knew that the techonology was revolutionary and would change the world. Thirteen years later we're still waiting on a massive transformation, and the delays to it are not remotely technical - people are just taking ages to adjust to a new understanding of their own interests.

Which implementation in python

2008-04-07T07:53:00.000-07:00

I was going crazy earlier not having which on Windows. Simple enough, but by posting it here I can copy and paste it when I next want it.


import os
import sys

def usage():
    print "Usage:"
    print "  %s SEARCH_STRING"%sys.argv[0]
    sys.exit(0)

if len(sys.argv) != 2:
    usage()

search_string = sys.argv[1].lower()

for path in os.environ["PATH"].split(';'):
    try:
        dir_contents = [fn for fn in os.listdir(path)]
        dir_contents = [fn.lower() for fn in dir_contents if os.path.isfile('%s%s%s'%(path, os.sep, fn))]

        matches = [fname for fname in dir_contents if fname.startswith(search_string)]
        for m in matches:
            print "%s%s%s"%(path, os.sep, m)
    except KeyboardInterrupt, e:
        raise e
    except:
        pass

Things to think about on those bad days

2008-03-30T15:43:00.001-07:00

http://www.daniel-lemire.com/blog/archives/2007/11/22/how-to-become-smarter/

Synergy tool

2008-03-25T09:18:00.001-07:00

There's a neat tool called synergy (handy osx client) that allows you to have a keyboard and mouse shared between computers. Once it is all set up, you can move the mouse off the side of the screen on one host onto another, and then keyboard and mouse actions take effect on that other host instead. Very useful! Another nice thing is that for the most part it Just Works. Even the clipboard data seems to carry reliably.

Note that it doesn't work properly if you try to use it from within vmware. I also had some niggles with the mapping of the windows/apple and alt keys between hosts being swapped. However, I've found a fix to this. In this example I'm using the 'meta' key mapping as a holding position while I swap alt and super about.

My keyboard and mouse are connected to a laptop running ubuntu linux. In the Gnome Keyboard Preferences pane within the "Layouts" tab, I have selected "Super is mapped to the Win-keys (default)". Then in the configuration file on the server side I have this arrangement:


section: screens
       ubuntu:
       osx:
        alt = meta
        super = alt
        alt = meta
end

section: links
       ubuntu:
        right = osx
       osx:
        left = ubuntu
end

From ubuntu I run this


$ synergys --config synergy.conf -f

Then I point the OSX client at the ip address.

reStructuredText and rst2a

2008-03-24T06:24:00.001-07:00

I've been wanting a quick document production system for a long time. Systems that combine format and data annoy me. Systems that are too complicated (e.g. LaTeX) aren't a lot of good to me either because if I try to use them it's only a matter of time before I can't use them at work. A contact pointed me to reStructuredText this morning. The combination of this and rst2a seem to be exactly what I need.

Curl usage isn't the most obvious thing in the world. Try this:


curl -F "rst=@text.rst" "http://api.rst2a.com/1.0/rst2/pdf?class=report&style=zope" > result.pdf

Storm ORM - problems from early steps

2008-03-22T10:33:00.001-07:00

I'm using the Easter weekend to write a network database for a Songseed subproject. I'm at the stage now where I need to bind between my layer and a relational database underneath. I thought I'd try out storm because although it looks underpowered compared to my current ORM of choice, sqlalchemy I've read only good things about it.

Unfortunately I've had several problems even very early on. My first attempt to install via ez_setup.py didn't work due to SSL complaints of some sort. I downloaded the egg directly for version 0.12. This did work.

Then when I try to declare a database as described by the tutorial I get this error:


>>> database = create_database("sqlite:")
Traceback (most recent call last):
  File "", line 1, in 
  File "/opt/local/lib/python2.5/site-packages/storm-0.12-py2.5.egg/storm/database.py", line 408, in create_database
    return factory(uri)
  File "/opt/local/lib/python2.5/site-packages/storm-0.12-py2.5.egg/storm/databases/sqlite.py", line 176, in __init__
    raise DatabaseModuleError("'pysqlite2' module not found")
storm.exceptions.DatabaseModuleError: 'pysqlite2' module not found

I could install pysqlite2 - but I'm not sure why I should need to. sqlite3 is now bundled with

I've tried linking to postgresql and that is behaving bizarrely as well - not all committed data makes it to the database.

So it's back to the cryptic world of sqlalchemy for the moment. this is an unfortunate result. I tend to get burnt with sqlalchemy because I find that it takes a lot of effort to get its workings loaded into my head (and doing this is crucial for avoiding bugs). Also, the storm documentation appears to be very good. Fortunately this is a modest project at the moment.

Update
Thanks to radix - as I finished this he responded to a message I'd left on irc. It turns out sqlite3 *wasn't* installed on my version of python 2.5 and I'd overlooked this. I have it all working in sqlite now and plan to continue working with it.

Intro and random thought about keyboard layout

2008-01-31T07:36:00.000-08:00

I used to have a blog at stable.cowoh.org but the server died as I was on a plane back to London after a holiday at my parents' place (where it lived under a desk). Terrible timing and I've been busy since and not had time to worry about it.

But today is my last day at DB and I'm anticipating that I'll have the next week to myself to work on songseed; play with some documentation ideas I have for the new arc language; do some server configuration work I've been putting off for ages; go to the gym and ... get back into geeky blogging. :)

Thought of the moment - one of the things that's annoying about lisp is that you have to press shift every time you want a parens. Since the numbers appear on the keyboard twice and I always use the keypad for accuracy, it would be nice to know if there was a way of customising my keyboard (will be a US English USB Apple keyboard after today) so that the default and shift cases for the numbers in the main section of they keyboard triggered symbol characters. Target OSs: Linux or OSX.