16 Mar 2014, 12:46

Saying goodbye to WordPress and hello to HarpJS

My blogging history goes back to 2001 when I setup a Blogger/Blogspot blog in the hope that I would update things more quickly than with Geocities or my Frontpage98 home page. 4 months later I wrote my second post :-) For a long time it involved writing the posts on Blogger and then publishing them via FTP to my own domain. It was fiddly but it worked ok.

Then around 2005 I discovered WordPress and realised blogging could be much much much easier. In the past 9 years I’ve done an incredible amount of stuff with WP. From personal blogging to building core parts of a business on top of it. It has been both a delight and the source of my greatest frustrations (glares at the “WYSIWYG” editor).

As with all great Open Source projects, the power of WP isn’t really in the core product itself, but in the ecosystem and community that built up around it. I have never run into a situation where I couldn’t find a WP plugin that matched my needs within a couple of minutes of googling.

The old slow, awkward, manually marked-up, manually-published HTML method died under the onslaught of WP’s ease-of-use. Bloggers no longer had to be programmers or web developers, they could be writers. And that was, without a doubt, a wonderful progression. Even tools that tried to straddle both worlds, like Movable Type, couldn’t compete with WP’s 2 minute install and one-click publish.

WordPress has also created a swathe of job opportunities for people around the world who do design, web-site creation, SEO etc etc, and who wouldn’t know a HTML tag if it slapped them in the face. This is both a blessing and a curse, as the number of security-compromised WP sites grows daily.

The incredible flexibility and architecture openness is naturally also WP’s greatest weakness. I’ve lost track of the number of plugins that either destroyed the performance of my WP installations or opened up giant gaping security holes. The relentless Core+Plugins upgrade cycle has also become insufferable if you run more than one site.

I’ve recently been thinking a lot around the lifecycle of things and how we often focus on the wrong bit. In App development, the creation of the App is the shortest bit of its lifecycle, so you should think about what you need to manage it, once its deployed. Similarly with blogging, the creation bit can take minutes but those posts can be read by thousands of people over several years. So we should be thinking more about how we make sure those posts are delivered quickly, reliably and with zero security hazards to as many people as possible and with no ongoing maintenance.

It was when we hit 2013 that I started to notice a change in the attitude towards WordPress. The WP features continued to pile up as it tried and failed to become a proper CMS. In reality most people want to create simple text+images blogposts with as little hassle as possible. So developers started going Back To The Future and using command-line tools, Markdown syntax, templating languages and, most importantly, GitHub Pages as the blog host.

This is the bit that started to get me excited. Totally static blog web-pages. No PHP, no MySQL DB, no possibility for security holes. And best of all, blazingly fast performance.

Of course these tools are currently completely opaque to non-technical people. You get ultimate control and flexibility but at the cost of having to learn Markdown, Jade, EJS, Stylus, Git, S3, JavaScript, etc etc.

As 2013 progressed I did lots more reading on these tools like Jekyll and Octopress. Somehow Ghost got lumped in there too but it’s not a static site generator, it’s trying to be WP with PHP removed and Node.js inserted.

My plan was to move some old work sites to be static. They have thousands of pages on them but despite little traffic, they regularly go offline as MySQL runs out of memory due to what appears to be some horrifically written SQL queries in WP or one of its plugins.

Finally a few weeks ago, I went at it all again but decided to use a blog move as a way of learning how it all works. I had a few requirements:

  • It had to work with GitHub Pages or similar
  • It should be git friendly
  • It should ideally use Node.js rather than Ruby or PHP
  • It should support more that one templating language/styling language and possibly CoffeeScript
  • The tools should run on Linux, OSX and ideally Windows
  • It should have a good community of people building themes and improvements
  • It should have some way of importing WordPress content
  • It should have some blog-style templates rather than web-site ones
  • It should have some support for commenting systems like Disqus, but ideally have its own JS commenting system

There are quite a few tools out there that support most of the above but I did find that many of them are either dead projects or single-developer projects with no community. The small set of which I found the most mentions, in no particular order, were:

A bit more digging bubbled up HarpJS as the most likely winner based on the above requirements, with DocPad as a backup in case it failed. I had also seen some Mobile guys blogging about HarpJS, and two of the co-founders created PhoneGap, which immediately piqued my interest.

It turned out to be a lot harder than I thought to move from WP to Harp. Wayyyy harder than the earlier move of that same blog from Posterous to WordPress. I’ll be writing a separate post right after this on the nitty gritty of the move but in this post I really just wanted to focus on the “Static Movement” (geddit? static, movement).

I really think there is a large opportunity for a mainstream Static Site/Blog platform and it many ways it’ll probably be like Blogger of old. I really like where Dave Winer is going with the Fargo outliner and it has the potential to be such a tool. The Harp guys are also doing some interesting stuff with the harp.io platform, but the entry price of $9.95 per month is probably too high for most non-commercial bloggers. The features an ideal platform would have include:

  • Publishing of the static content to GitHub Pages, Dropbox and Google Drive via whatever APIs are appropriate, including Git
  • Dropbox and Google Drive to provide CNAME support for these static sites
  • Built-in hosting alternative like harp.io
  • Command line flow using git etc for developers
  • Online management/publishing dashboard for non-developers
  • Online Markdown editing tool - I think Markdown could also go mainstream. Anyone can learn it in a few minutes.
  • A split-screen online Markdown editor like Mou or Stackedit
  • Online templates/themes management using something like Jade+Stylus or EJS
  • Very fast compile/publish cycle
  • Plugins only available as JavaScript snippets (Dropbox JavaScript “App” support is very cool)

I just have this feeling that Dropbox/GoogleDrive could finally be the trigger that gets us back on the righteous path of people owning and publishing their own content instead of giving it for free to Facebook/Tumblr/Twitter where it could all disappear tomorrow (cf Geocities/Posterous/etc).

I’m probably a week or so away from pulling the trigger on the move. Some of the earliest content that originated on Posterous is well and truly messed up and I’m not sure if I want to script up some cleanups or leave it. Finally, the lack of pagination is probably the last thing I want to look at before making the DNS change.

It’ll be interesting to watch how 2014 plays out. Will a new platform come to the fore which focuses on ownership, content portability, zero-maintenance, security and adding a bit of fun back into blogging?

14 Mar 2014, 20:50

Running Debian Linux, GCC, Emacs, Git, Node.js, npm and HarpJS on Samsung Galaxy S4. Oh my.

I’ll shortly be moving this entire blog from WordPress to HarpJS generated static pages over on GitHub. It hasn’t been as smooth a process as I’d hoped but I’ve learned lots. We’re also using HarpJS in a work context and it works great there running under Node. I’ll write it all up either this weekend or next.

Then this evening I asked myself the silly question “I wonder can I run Node.js on Android?”. It turns out you can. But it’s a teensy bit fiddly to get going. The biggest problem turned out to be Android 4.4 KitKat which made the first few steps the hardest.

Debian on Android

You are going to be running Debian Linux in a chrooted environment. For some weird reason I never even knew this was possible until today! In fact there are multiple projects out there that enable this. The main thing, of course, is that your phone must be rooted. A Stock Samsung ROM is fine and I’m sure GPE and Cyanogen would also work.

I initially tried “Linux Deploy” from the Android Play Store but hit a brick wall with it mounting the external SD card on the S4. I then tried Linux On Android, also from the Play Store and hit similar problems, so I dug a bit deeper and got to the bottom of it.

All of this is easier if you connect up an external keyboard to your S4. I’ve found that a cheap USB OTG cable works brilliantly for thumbdrives, mice, keyboards and even the 2.4GHz combo keyboards you get with Android TV sticks.

find this bit:

elif [ -d /Removable/MicroSD ]; then
echo "mounting /Removable/MicroSD to $mnt/external_sd..."
$bbox mount -o bind /Removable/MicroSD  $mnt/external_sd

and change it to the following:

elif [ -d /storage/extSdCard ]; then
echo "mounting /storage/extSdCard to $mnt/external_sd..."
$bbox mount -o bind /storage/extSdCard  $mnt/external_sd
  • Save that
  • Then run the Complete Linux Installer App which mainly consists of instructions. The only important step is to pick Debian in the drop-down list and change the settings so that the image directory is set to where you copied those image files to e.g. /storage/extSdCard/debian
  • Then just tap the Start Linux button
  • You may see some initial errors but if all is basically ok, you’ll get a first-time-run set of Debian questions asking for password etc.
  • And you are done from a Debian perspective. You can even setup the VNC server to have a basic UI but I haven’t got that working properly yet (blank screen with X pointer).

So now that we have an almost bog-standard Linux setup, you can pretty much do what you normally would. I have run into some small glitches but found workarounds quickly for them all. I should probably setup a non-root user at some point too. In any case, this is how I setup HarpJS:

apt-get update
apt-get install nodejs nodejs-legacy
apt-get install git curl build-essential openssl libssl-dev
wget https://npmjs.org/install.sh
chmod 755 install.sh
./install.sh
npm install --unsafe-perm -g harp
mkdir gitwork
cd gitwork
git clone https://github.com/conoro/conoro.github.io.git
cd conoro.github.io
harp compile _harp ./

The two bits of oddness above were getting an error from curl trying to install npm, so I used wget. Also, very very strange node-gyp permissions errors installing Harp globally when it came to node-sass (hence the –unsafe-perm). Node-sass also causes problems on Windows I’ve noticed.

Harp on Android And that’s it about it. Except for…….

apt-get install emacs

How could I not? :-)

Emacs on Android

So I edit Markdown blogposts in Emacs, compile with Harp and commit to GitHub to publish. Simples.

13 Mar 2014, 08:35

Upgrading your Samsung Galaxy S4 i9505 to KitKat 4.4.2 on OSX

Using an S4 on a Mac Book Pro is an exercise in futility now that it and other phones no longer appear as a disk drive but as a weird media device. The Samsung Kies software doesn’t help and actually seems to make things worse. If you run Windows 7 inside VirtualBox you can see the phone’s storage but doesn’t see it when the phone is in upgrade mode. So you can’t run the usual Odin software there.

Then I found out about Heimdall, an open source tool for upgrading Galaxy phones on Windows, Linux and OSX. It looks scary and the UI is a horror but once you find out the command required, the whole thing only takes a few minutes.

So assuming you want a Stock Android 4.2.2 ROM (rooted of course) on your phone and you are coming from some other ROM like the Google Play Edition ones, then the following will do the trick on An MBP.

  • Uninstall Kies if it’s installed. On a Mac, you stupidly have to re-download the Kies installer to get to its uninstall routine. Oh FFS.
  • Download and install Heimdall
  • Download the new 4.4.2 ROM. I used this one
  • Rename that file from .rar.md5 to .rar
  • Install an Un-Rar tool like UnrarX
  • If you are coming from a wildly different ROM, you’ll probably have to Factory Reset. Before you do that, backup your Apps/Data using Titanium Backup and make bloody sure it backs up to SD Card, not internal memory (unlike me!).
  • Power-off the phone and re-boot it by pressing Volume-Up and Power. When the bootloader menu appears, pick the factory reset option and then power-down
  • Un-rar the downloaded file and you’ll get a directory with a bunch of files in it.
  • Put your S4 in download mode by shutting it off, then restarting it by holding Volume-Down and Power. At the prompt, press Volume-Up. Then connect it by USB to the Mac
  • Open a terminal in OSX, cd to the directory that you un-rarred and type the following
sudo heimdall flash --APNHLOS NON-HLOS.bin --ABOOT aboot.mbn --BOOT boot.img --HIDDEN hidden.img.ext4 --MDM modem.bin --RECOVERY recovery.img --RPM rpm.mbn --SBL1 sbl1.mbn --SBL2 sbl2.mbn --SBL3 sbl3.mbn --SYSTEM system.img.ext4 --TZ tz.mbn --CACHE cache.img.ext4
  • That’s it, after a few minutes you’re done, the phone will reboot and give you an all clean setup.
  • Optionally Run Titanium Backup to recover your Apps/Data

09 Mar 2014, 18:39

With the Enigma 2 software, the Sab Unix HD Triple Tuner is a very impressive satellite/terrestrial box

In addition to the recent Andro100 box, the guys in TV Trade also gave me a Sab Unix HD Triple Tuner box to try out. The specs on this thing were pretty amazing for something that costs approx €130+VAT so I was intrigued to see how well it did. After the reliability horror that was the Ferguson Ariva 120 pile of junk, I went in with open eyes.

Sab

The box itself is pretty compact with a 1980s Amstrad stereo vibe off it with the chrome accents. Not exactly a looker but not offensive either. It has dual satellite inputs and a single “Saorview” DTT input. The usual outputs like HDMI etc are there along with wired LAN and a front and back USB port. I hate those front USB ports on all boxes as they are behind a flap which you then have to leave open if you use the port. But if you want to insert a wifi dongle, it probably makes sense for signal strength.

I connected up a 1TB externally powered USB harddrive to the back and stuck a generic nano Wifi dongle in the front. Both were recognised immediately and I was able to connect to the home network and format the drive as EXT4 within minutes. The box is Linux under the hood which is why stuff like this just works. The reason I went with EXT4 is that I did some tests recently on our home server and EXT4 write-speed is 3-4 times faster than NTFS on a Linux box.

The general UI of the box wasn’t great and I found the whole EPG setup to be annoying and overly complex. However I was easily able to setup the nightly internet download of a 7 day programme guide so the lack of Freesat certification was not a problem. This is a huge advantage over cheaper boxes.

We used the box for a few weeks but continued to struggle with the generally clunky UI, awkward EPG and remote control. For example, the pause button on the remote control doesn’t do time-shifting, it does freeze-frame. Who has ever wanted a freeze-frame button? You have to hit the time-shift button and then hit pause. The box was also unable to record shared channels like CBBC/BBC4 which was infuriating for Bron/Broen/The Bridge.

The software had a ton of completely useless features like “apps” for various German stations. The apps felt like the old Unison boxes from 10+ years ago.

These usability problems meant our usage tailed off. But the guys in TV Trade mentioned that there was an alternative opensource front end for the box called Enigma 2 and that I should give that a whirl. They had all the files on their site plus instructions and video. It was actually a doddle to install and took less than 10 mins.

I couldn’t believe how good Enigma was compared to the default software. Infinitely more usable and slick and just felt far more “modern”. However it too suffers from an EPG that just takes too much work to figure out. It has a kitchen-sink set of features when all I want is a list of available channels and the ability to say “include/exclude in TV Guide”. All this stuff with Bouquets is straight from the old MPEG specs and should never be seen by an end user.

I was starting to get frustrated with the EPG setup and did some googling to discover that there is a PC app called dreamboxEDIT which lets you configure the EPG remotely. I was pleasantly surprised to find that it was very easy to use and I simply deleted all the “bouquets” and removed all the garbage channels so I just had the core Irish and UK ones.

EPG

So yes, it was fiddly to get that sorted but we now have a really nice program guide with 7 days of content. You can’t do series link like Sky but you can set repeating timers for the same time.

We have played with dual simultaneous recording and it works very well which is huge relief given how bad the Ariva was at even recording one channel. Time-shifting/pausing also works well but takes a few seconds to kick in.

The difference between the default software and Enigma is like night and day. I’ve noticed that we are watching live satellite TV far more often now that it’s easy to do.

The box also has a media player which can play network videos from your PC or NAS and you can easily FTP to it too. I’m having very slow FTP transfers off the box but I haven’t investigated that properly.

I’d recommend that the TV Trade guys ship Enigma by default with a single default Bouquet of the “normal” channels pre-configured. If they did, the Triple Tuner box would be usable by most people from the get-go, no matter how non-technical. As it stands, it’s brilliant value but you need to do some work to get it setup just so.

It’s great to be finally back able to watch BBC2, BBC4, Channel 4 etc again (and not forgetting GirlPop for our youngest). The timing of getting the box was also perfect for the Winter Olympics and we watched a ton of it.

If you have cable cut and you are looking for something powerful but inexpensive, the Sab Unix HD Triple Tuner is well worth a look.

Excuse me whilst I go set the timer for Salamander on BBC4 tomorrow.

28 Feb 2014, 17:34

My Arduino and Raspberry Pi links for newbies

I generated this list a few months ago for Clonakilty Coder Dojo after demoing a pile of my projects/gadgets to them. Can’t believe I’m only blogging it now. Hope you find it useful.

Learning about Arduino

Making Arduino and Raspberry Pi work together

Gert van Loo who did the original design for the Raspberry Pi now has an RPi board called the GertDuino that contains an Arduino and can use Arduino add-on shields. It seems like the best of both worlds and I’ll be getting one for myself.

Learning about Raspberry Pi

Raspberry Pi Minimum setup

Raspberry Pi + 4GB SD Card (Class 10) + 1.1 Amp Phone Charger + HDMI cable + Wireless KB/Mouse + Wifi Dongle It can work with old TVs on SCART cable but picture is very fuzzy.

Very cheap Chinese Electronics including Arduino clones

The main site is DX.COM - They have tons of stuff. Very cheap. No postage charges but takes a minimum of 3 weeks (usually more) for things to arrive. Make sure to keep orders less than approx €50 to avoid import duties. Just split large orders up. Assume that if things break, you’ll either repair or bin them. Doing returns to China is a disaster.

Elecfreaks in China is also very good.

###Other Arduino Starter Kits * Official Kit * Alpha Crucis * Cool Components * SeeedStudio

US Suppliers

Sparkfun and Adafruit are the main ones. Can be pricey to buy from US due to postage and import duties.

UK Suppliers

Cool Components are good but postage can be pricey

French Suppliers

Alpha Crucis seems like good value, particularly when Euro weak, but I haven’t used them yet.

Irish Suppliers

Farnell has a large Irish warehouse. Incredible selection of components mainly targeting Trade. But cheapest way to get Raspberry Pi in Ireland. They have a minimum order of €20 but then deliver by courier which is good. If you need to bulk buy wires, switches, ICs etc, they are the best/fastest option.

Makey Makey

Available on their own site and on Sparkfun in the US and Cool Components in the UK

Bare Conductive Ink

On their own site in the UK.

Maplin RPi Kit

Everything you need in one box on the Maplin site.

They had them in Maplin Blackpool the last time I was there (opposite Argos). Approx €95. Not cheap but you are guaranteed that all the parts will work together. The RPi is particularly finnicky about USB devices and power supplies.

i-racer

From Cool Components in the UK or Sparkfun in the US.

Remote Control Car Batteries

Hobbyking is best for these type of batteries. Hong Kong store has biggest range but obviously longest delay. UK store has quite a bit of stock but charges a lot for postage. I use Parcel Motel to order from UK store at UK postage rates and then collect in Bandon:

15 Feb 2014, 14:09

How hard can it be to make a simple LED blinking circuit? Conor's stupid questions to self, Episode 5892

#“How hard can it be to make a simple LED blinking circuit? Conor’s stupid questions to self, Episode 5892”

Last night after a few glasses of vino, I decided to make a simple blinking LED circuit but I didn’t want to involve Arduinos, just discrete components.

Most of that auld internet does them with 555 timers but I don’t have any of those so much googling ensued. I finally found one and tried to make it, learning in the process that soldering and Sauvignon Blanc don’t mix:

blink1

 

To be fair, the LED lit, it just wouldn’t blinking blink.

I’ve written several times before about how an infamous lecturer in UCD put me off analogue electronics almost for life, so after a bit a clueless fiddling I went to bed.

I tried again this morning and found what appeared to be a nicer circuit and I even had the right type of transistor:

led_flash_blink_valentines

A bit of fiddling later:

blink2

Next year I’ll just buy a card in time!

blink3

I even did a video for the full blinktastic experience:

12 Feb 2014, 21:20

Fabulous printable geeky Valentine's Day labels from Evil Mad Scientist

#“Fabulous printable geeky Valentine’s Day labels from Evil Mad Scientist”

Dare ya. Click image for full set. Best company name of all time.

GeekValentines_-_GeekValentines_pdf

 

02 Feb 2014, 14:12

Was anyone else a member of Fireball's Club from the 1970s Bullet comic?

#“Was anyone else a member of Fireball’s Club from the 1970s Bullet comic?”

It was huge in our school. I even had the special plastic medallion/key. For some reason I managed to block out his ‘tache in my memory. Here are two pics from Google Images which match my own experience.

03 Story

4907766957_165ce2990dSecret codes and invisible ink were major parts of my youth. I wonder if we could use something similar to get kids interested in privacy and crypto ;-)

 

 

01 Feb 2014, 16:13

Run the @espruino JS interpreter on Windows

#“Run the @espruino JS interpreter on Windows”

I just got my Kickstarter Espruino yesterday. It’s one of those projects that just makes me smile. A tiny board with an ST ARM MCU running a special JavaScript interpreter that can do all the stuff with hardware you would normally associate with C/C++/Assembler/Wiring.

espruino_board

I’ve only just got started playing with it and I swear it feels like my first days using a ZX Spectrum. The instant feedback of interpreted JS in a console feels exactly like typing BASIC into the Speccy (blog post coming on my Speccy and the Interface1bis soon).

The Espruino Chrome App is pretty basic but I was really excited to see that it has a Scratch-style blocks-editor for generating JS. I love the whole JavaScript/Node-everywhere movement but all those damned brackets/braces are a real hurdle for anyone trying to learn programming.

espruino

The Espruino interpreter itself can be compiled for a variety of ST Eval boards, the Raspberry Pi and most boxes that run Linux. I’d love to get it running on my DigiX too since its processor specs are almost identical to the Espruino. But it’s been over 14 years since I played with BSPs and cross-compilers and I’m not sure I have the chops any more.

Gordon, the man responsible for it all, mentioned in the forums that it should compile on Windows using Cygwin. Speaking of 14 years ago, it’s about that long since I used Cygwin too. I spent so many hours in the late 90s building XEmacs for Cygwin on my PC.

And whaddya know, it works! It took forever to download all the necessary bits of Cygwin on our crappy DSL but compilation of Espruino took about 30 seconds with no errors. Here’s some code in action which grabs the contents of the Espruino web-site home page:

http

So if your curiosity is piqued by this very cool project, you can play with the core interpreter on Windows whilst you wait for your board to arrive. I was chuffed to see that the http functionality works fine.

I built it on Windows 7 64-bit so I’ve no idea if it runs on 32-bit Windows. You should just need the EXE here and the Cygwin1.dll hereto try it out. Let me know how you get on. I’m not 100% sure yet if you need any more of the Cygwin DLLs to access other functionality.

 

26 Jan 2014, 19:43

The AND100 Android Box is perfect for Netflix, RTE Player etc

#“The AND100 Android Box is perfect for Netflix, RTE Player etc”

David in TV Trade very kindly gave me an AND100 Android Box before Christmas and I’m only now finally getting around to reviewing it.

Clipboard-1

I’ve been keeping an eye on the whole Android TV Stick/Box market for a while. There are so many out there, it is almost impossible to pick one and I haven’t been overly impressed with the sticks due to all the compromises involved in getting everything into such a small form factor. Wifi performance in particular seems to be a failing.

This AND100 box is a different kettle of fish. It’s a box for a start! It also has wired Ethernet as well as built in Wifi. It comes with a couple of USB ports so you can add a Wireless Keyboard and USB harddisk if you wish. In addition to the usual HDMI, it also has Composite out if you have an older TV.

The only real difference from standard Android 4.x is that they’ve added a pretty hokey home page which I could really do without. But so far that’s the biggest complaint I can make. It has full Android Play Store and it was a doddle to install Netflix, RTE Player, iPlayer etc.

Android hasn’t been optimised for non-touch devices so I was surprised how well the simple IR minimal remote control works. Up/Down/Left/Right play/pause etc is really all you need for most of the Media Apps. Netflix works particularly well compared to the PC version I’m used to. The extra wireless keyboard with touchpad deals with those times when you need text input and you find the on-screen keyboard is too much hassle. In an ideal world both remotes would be merged.

All of the media Apps I tried like Netflix worked like a charm which is hardly surprising since it has a dual-core 1.5GHz CPU. The only let-down was XBMC which still doesn’t have hardware h264 decoding on most Android devices and struggled to play 720p stuff across the network. But that is more down to XBMC and I hope to see that sorted for most devices in the coming months.

Whilst you can get Roku boxes etc to play Netflix, they are extremely limited and obviously don’t run Android. In fact they currently don’t even have YouTube. And whilst I love XBMC/RaspBMC on Raspberry Pi, things like YouTube take an eternity to load.

I haven’t tried any games on the box so I don’t know how the UI would work but there is nothing stopping you from giving it a lash. Whilst I almost never watch RTE, I did try out the RTE Player and it worked perfectly.

Overall, it’s a great piece of kit and it should be a trivial exercise to install OpenVPN and connect to your VPN provider so you can access the likes of local BBC or Hulu on it.

The price is actually great compared to many of the Android sticks out there which are far less functional. You also know that if anything goes wrong, you have a local supplier and you won’t have to deal with the black hole that is returns to China.

Now if only House of Cards Season 2 would hurry up and get here.