2011 in review

The WordPress.com stats helper monkeys prepared a 2011 annual report for this blog.

Here’s an excerpt:

The concert hall at the Syndey Opera House holds 2,700 people. This blog was viewed about 10,000 times in 2011. If it were a concert at Sydney Opera House, it would take about 4 sold-out performances for that many people to see it.

Click here to see the complete report.

Advertisements

FCP X Parody Ads

Inspired by this Parody FCP X ad, but realising I’ll probably never make them at all, and certainly not while it’s still topical, I decided to script up my two ideas for an FCPX “I’m a Mac/I’m a PC”-style ads and release them under Creative Commons so someone with more time could make them (you know, if they wanted to).

Being freed of the need to be able to shoot them, the second one got a bit elaborate. The first one is quite short.

FCPX Parody ads

Preliminary notes on Apple malware protection in an enterprise context

(As a lot of the search traffic coming here seems to be trying to find out what XProtectUpdater is, let me answer that: it is the agent installed by Apple as part of the 2011-003 security update. It handles downloading new “signatures” for apps which should be regarded as malware)

For the first time ever, we have an official Apple malware protection mechanism (and also some actual malware, although it is totally human engineering and requires gullible admin users and it only tries to get their credit card number, so the actual infrastructure damage is nil).

But the threat’s there, and so is the fix, and a responsible admin user should look at deploying it. See: http://support.apple.com/kb/HT4657 (which links to several more pages which contain actual info)

In short, we now have a new mode for the “you’ve never run this file before” warning (aka File Quarantine) where known malware elicits a “don’t run this, trash it!” warning.

Malware dialog (deep-linked from support.apple.com)

To go with that we have a malware definitions file and a means of updating them (at some unspecified interval).

But of course, as-is it’s all single-usery and it runs an auto update which may not be appropriate in, e.g., some critical on-air or video editing contexts. And it talks to the outside world via some protocol. So, wanting to deploy it in a managed way, I dug around and found out the following:

Auto updates do not work via a preference plist, the security update installs a launchd item and enables/disables this (via the overrides db à la launchctl unload -w) as you toggle the preference.

The launchd job is called com.apple.xprotectupdater and itself calls /usr/libexec/XProtectUpdater . The interval is every 24 hours (every 24 hours after it’s run, not at any particular time).

XProtectUpdater appears to talk on port 80 and know about system proxy settings (including .pac files). It talks to the proxy and returns 0 if run when I’m on the corporate LAN and 255 (with an error message) if I have the ethernet unplugged. At the Apple end, the definitions file lives here:

http://configuration.apple.com/configurations/macosx/xprotect/1/clientConfiguration.plist

There are defs for quite a few nasties already!

Looks like the defs are locally stored in /private/var/root/Library/Caches/XProtectUpdater/Cache.db, an sqlite3 database. So you could simply push that file out, though that seems more likely to break in the future – but it may be your only choice.

So, it appears in an enterprise situation you can:

  • install the pkg in the background via ARD (or shell)
  • enable or disable the auto update by calling sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.xprotectupdater.plist (or launchctl load etc. to enable)
  • Manually trigger an update of definitions by calling /usr/libexec/XProtectUpdater (probably with sudo).
  • You could also manually update by enabling the launchd job (it runs immediately) and then switching it off a few minutes later. This might be less likely to break in the future
  • Presume if you have ports open to configuration.apple.com (or *.apple.com) already, this will just work.
  • Or, you could manually push out a new Cache.db file (permissioned correctly) as and when you saw fit.

I’ve yet to do any of this and you should, of course, only try it yourself if you understand what it means and are ready for anything unexpected.

Update:

After getting the more-than-unusually cryptic error: “An operation failed in launchdadd for reasons that you probably can’t do anything about. Maybe you should reboot.” I’ve found that: if you try to change the “auto-update” setting more than 30 seconds after you open the prefpane, it will fail to actually take effect .

Hope that helps someone!

Half-arsed postal-code-derived location info on websites (rant)

The Post-Code (“ZIP”) I live in covers three actual suburbs – and while this is certainly not the norm, neither is it in any sense a freakish one-of-a-kind geobureaucratic calamity.

Tonight, smh.com.au popped up a little (by their standards very unobtrusive) window saying if I wanted more accurate weather info I could enter my postcode. Now, over the years a considerable number of websites have offered to simplify my whole “input where you are” experience with the “just enter your post code” text box. And for a lot of people, it probably is nice and simple. But the experience I had with smh is all too familiar, and tonight I rail against it!

I entered my postcode – 2044 – and after a second or so of AJAXian information transfer, I was asked – yet again – whether that meant Sydenham, St Peters or Tempe.

Really, Fairfax, not only is the weather for adjacent suburbs perceptibly different, but you actually have the separate forecasts for them? Call me cynical, but I don’t think so.

In fact, I can’t think of many times when the specific suburb has actually mattered – obviously when you enter your whole address it does, but for “find your nearest outlet” or various real-estate searches, or a whole host of sites where the actual suburb does not matter in the slightest, the “simple” process of entering 4 digits turns into a frigmarole. I can’t find documentary proof, but I’m sure at least once I’ve gone through this for something that only needed to know what state I was in!

It happens so often I can only assume some code snippet from the intertubes is getting used and reused. And, FFS people, it could use some refinement!

If you want to reuse code snippets, why not ask me to OK a Google geolocation lookup? That will give my location to within about 20m – surely accurate enough for even the most fine-grained weather forecast?

Quick Notes on Magic Lantern for 550D

UPDATE: work has progressed rapidly on ML for 550D, you should head to magiclantern.wikia.com/wiki/550d for the latest, or go to the bleeding edge at groups.google.com/group/ml-devel/

i.e. – probably not worth reading what’s here …
(ENDOFUPDATE)

I’ve just got this working (my camera is not bricked!), my forum membership has not been granted, so I can’t update the various wikis. Instead here I’ll just detail what I had to do to.

  1. start with the wiki installation notes
  2. Where it says “newer autoexec.bin available” – download it, but bear in mind that the manual gain does NOT work, due to a small bug. Until I can get a new build on the wiki or whatever (GPL means I can’t conveniently distribute my build), you’ll have to build it yourself. More on that later
  3. The OSX auto-boot notes at the end are accurate, but incomplete. You will need to have the card unmounted but not ejected. You do that via Disk Utility, which will also tell you whether your card’s FAT16 or FAT32. To find out whether it’s disk1s1 or not, use diskutil on the command line
  4. To un-make the card bootable under OSX (where you don’t have CardTricks) simply rename it. Without a bootable card called “EOS_DEVELOP” your camera is exactly as it was before.

You should now have a bootable card which display magic lantern message, slightly redundant extra ISO/shutter info and, crucially, you get on-screen audio meters. But you still get audio auto-gain.

To defeat the auto-gain, this thread mentions a new autoexec.bin but if you read to the end, you’ll see it’s been discovered that the name for the config file, “magiclantern.cfg” is too long for the 550D’s brain. You need to shorten it to an 8.3 filename, and put that name into the 5d-hack.c file (e.g. I went with “magilant.cfg“, others favour “magic.cfg“), then build your own. Again, there are build instructions but I had to deviate from them. My notes are below, they’re terse and this will only work if you have some idea of what you’re doing:

• need newer gnutar to run summon-arm script (and make script use it)
• need to get mercurial (via MacPorts)
• makefile patch out of date. Only had to: modify arm path, gcc version (I get 4.5.2, they’re talking about 4.3.2)
• link problems – zero-byte .lds file causing trouble. deleted it. got rebuilt

But the reward is this: audio meters onscreen and silence when there should be silence.

own build of ML on my Kiss X4

Compressing some js for the js1k comp

Lately there have been both a 1k and a 10k JavaScript competition. The 10k one is wayy too fancy to appeal to me, but I thought I might have a stab at the 1k.

In doing so, I made a few discoveries about methods to get your js code really small.

I saw this tetris entry which achieves ALL of tetris in 1024 bytes of JS. Goddamn! At first glance, the source is bizarre gobbledygook, but once you study it, it’s actually doing “real” compression: putting recurring bits of text into an array and substituting them into the main code (by searching replacing chars which were not otherwise used in the code). This was obviously done algorithmically, but it occurred to me you could do it manually and use slightly meaningful unicode chars, and end up with dense, quick to write yet readable code – and you didn’t have to compress it afterwards, it was ready to go (I did, in fact, write it with linefeeds in, but I didn’t have to do much)

e.g. you could replace the text “function(a,b,c)” with “ƒ”, and then just have myFunc=ƒ{alert('blah!)}. There’s enough whacky symbols out there in unicode to cover a lot of normal long words – I used ® for “return”, ß for “button”, £ for “EventListener” etc. A quick sample:

d=ƒ{if(e.ß!=undefined){stkTo(X,Y);nuPth(X,Y)}else{up}};\
↓=ƒ{nuPth(X,Y);D.add£('ⓜmove',d,0)};\
up=ƒ{D.remove£('ⓜmove',d,0)};\
t.lineCap='round';P();\
c.add£('ⓜ↓',↓,0);\
D.add£('ⓜup',up,0);\

I managed to fit the first version of my sketchy app into <1k this way. Check out the demo source – it's at least partly readable. Then, I managed to trade off a little more readability and get a version with undo.

But THEN, @getify submitted an entry in the comp which actually did @aivopaas’ compression for you. As I still had a feature or two I’d really like to include, I decided to end my readability experiment (but write it up somewhere – i.e. here) and go for maximum smallness, and here are my notes on achieving that.

The idea behind @getify’s compressor is you first minify it using some other tool – it doesn’t strip whitespace, shorten tokens or any of that jazz – and then compress it.

My code was already at a slight advantage because it contains repetitions where they don’t add functionality, but they do lend themselves to getting substituted out.

Starting with my “decompressed” code (i.e. code in the form I never actually typed it) I first tried Google’s closure compiler (just the web service, I didn’t build it myself or anything), but that had three problems.

First, it converted all my quotes to doubles (see footnote) which @getify’s code just passsed thru and it all broke – but I could fix that manually.

Second, it still put two linefeeds in my code – again, once I worked that out, manually fixable.

Lastly, however, the Closure compiler “fixed” my function declarations to have the right number of parameters – a trick I copied from a few js1k entries is to re-use the same function dec regardless of how many parameters the function might use, thus being able to use the one compressed token for all of them (the ƒ above).

Then I tried jsmin (again via a web service). While its compressed code was maybe 20 bytes (a whole 2%) larger than Google’s, the result out of @getify’s was smaller. My hand-squeezed 1022 bytes was now 954. But I could see that there were still improvements: @getify’s had separately tokenized “=function(e,x,y){” and “=function(e,x,y){t.” – where the former could be substituted into the latter to save 15-odd bytes! Also, the compressor, very cleverly, uses the ASCII characters below 32 as tokens so as not to have to use unicode chars (which are at least two bytes). The downside is, it can only have 31 tokens, so it picks the 31 best, but leaves room for improvement. Now, there are plenty of unused roman letters and symbols – Q and Z don’t appear, so as I added my remaining features, I added a few more hand substitutions to try to keep me below 1024. There are still a couple of repetitions (the word “button” the most obvious) and a couple of other tricks I used when I did then hand compression, so I know @getify’s could be improved and I wonder whether, once improved, it could get itself back under 1k again!?

But there’s another point here which is that, while it’s a very neat trick, there’s been just a little more development put into gzip, et al. I vote the next js1k comp limits you to 1k after gzipping. it would still reward tricks like repetitive function decs, but free us up from trying to reinvent the wheel, and including that wheel into our 1024 bytes.

I ended up fitting 2398 bytes into 1022 – admittedly that was starting with comments, which broke down like this:

Original code   JSMin’ed   @getify compressor   hand tweaks
2398 1442 1044 1022

The entry is now up, and here it is in action (again, owing a little something to those who came before it)

skech app doing a van gogh

skech app doing a van gogh

Update: Aivopaas releases his compressor, found a good minifier comparison page

@aivopaas has now put his JavaScript compressor up – it does a better job than @getify’s, as expected, and would reduce my entry to 988 Bytes.

I also found CompressorRater which runs your code thru 4 different compressors and lets you choose the one that does the best. (However my code, which has raw html output by document.write() calls, got mangled, at least I knew there was only a byte or to between the main ones)

Footnote on why you want only one kind of quote:

the way the compression works is to declare one big string that gets substituted into, so you want to use only one kind of quote in your code so you can quote it with the other – e.g. this is one string

"alert('dog'+'cat'+'ferret')"

which you can then eval(), if the quotes were all the same, you’d get a parse error.

The Rise and Fall of .. things

In the past week, I’ve read three separate peices comparing either the US, the “Developed World” or broadcast TV to the Roman Empire just before it fell (well, I skimmed a few, there was a lot of detail). Finding parallels is a fun game to play, but I couldn’t help thinking:

I read them in the Roman alphabet, over two months with Roman names, most of the words over two syllables were Roman (well, French) and my state Premier and Federal Opposition Leader regularly consult with representatives of the Roman Pontifex Maximus.

So, provided they’re prepared to change a little, uncle Sam, the first world and the networks might not have so much to fear.

(BTW here are two of the articles:

)