Category Archives: Programming

Lego Mindstorms EV3 and ev3dev

So recently my Lego infatuation led, as it inevitably would, to Lego Mindstorms.

This post discusses a lot of the setup I’ve done to get the EV3 controller brick set up for development using a “normal” programming language instead of the visual one provided by Lego. That does look to be a nice way for a lot of people, especially kids, to learn to program robots. However, as a guy who has spent a very large number of years programming, visual languages like that are a bit confining. So imagine how happy I was to see that the ev3dev project has stepped in! (More about that later in this post)

So now I have a pretty functional development environment set up – time to work on some actual robotics!

Topics covered:

  1. Remote Editing
  2. RSA Key for SSH
  3. FTP Server
  4. Python 3

While some of these such as remote editing are Mac-specific, a lot of the content here is the same regardless of your host computer.

About Ev3dev

Ev3dev is an awesome replacement OS for the EV3 brick, available here: ev3edev.

I won’t duplicate a lot of detail that’s on their website, but in general, it provides an alternate OS that lets you write Mindstorms programs in a variety of languages. The one I’m using is Python. The entire project is really slick, and an impressive accomplishment.

The ev3dev build I’m using is Jessie 2015-12-30.

All of the steps here assume that it is installed and running, and a network connection has been established along with being SSHed into the brick. (All of that is covered on the ev3dev website.)

Disclaimer: I have not gone back to a clean install of ev3dev and run though the steps in this post, so while I believe I captured everything, it is certainly possible I missed something.


Enable remote editing with TextMate

This obviously assumes that you have the latest TextMate running on your Mac. Did I mention that this post is going to be Mac-centric in places?

On EV3:

$ sudo gem install rmate

On Mac:

Add this to your  ~/.ssh/config file:

Host *
RemoteForward 52698 localhost:52698

Or, when starting SSH, you can do this and not muck with the SSH config file:

$ ssh -R 52698:localhost:52698 robot@ev3dev.local


 Setup RSA key for SSH

By setting up the RSA key for my host computer, I can skip needing to enter a password every time I log in via SSH.

On the Mac:

$ scp ~/.ssh/id_rsa.pub robot@ev3dev.local:/tmp

This assumes that an RSA key has been previously generated! You can generate a key using the ssh-keygen command, which will create the private and public keys in (typically) the ~/.ssh directory. The file without the extension is the private key, and the one with the .pub extension is the public key.

This public key is the one transfered to the system you want to access, and append to its authorized_keys file.

On the EV3:

$ mkdir ~/.ssh
$ cat /tmp/id_rsa.pub >> ~/.ssh/authorized_keys

 


FTP Server

Even though I can edit files via rmate and sip them back and forth from the remote shell, it’s nice to be able to move batches of files etc. via FTP. And with something like the Transmit application as an FTP client it’s trivial to do a sync and essentially back up my EV3 home directory to my Mac.

I spent a stupid amount of time trying to get vsftpd installed and running (installation went smoothly, but running always failed with an error no matter what I tried. And I tried  a lot of things!)

So, since my goal is just to have simple FTP access,  on to plan B – use Python! I grabbed a very nice FTP package for Python here: https://pypi.python.org/pypi/pyftpdlib/

The following shows the steps I went through (you may want to update the wget line based on the current version of pyftplib). In a long-ago effort to install this library on a Beaglebone, the installer failed, so the steps below include just manually moving the library into place – didn’t want to invest a lot of time debugging that issue since manually moving it seemed to work just fine.

Install the FTP package

$ wget https://pypi.python.org/packages/source/p/pyftpdlib/pyftpdlib-1.5.0.tar.gz
$ gunzip pyftp*
$ tar -xf pyftp*.tar
$ sudo mv pyftpdlib /usr/lib/python2.7

Create the FTP Server Python program

I added a super-simple FTP server program to my home directory ~/robot.

#!/usr/bin/env python
from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import FTPServer
from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import FTPServer

authorizer = UnixAuthorizer(rejected_users=["root"], require_valid_shell=True)

handler = FTPHandler
handler.authorizer = authorizer

handler.abstracted_fs = UnixFilesystem

address = ('', 21)
ftpd = FTPServer(address, handler)

ftpd.serve_forever()

This will allow you to log in using any account other than root. By default, the username of robot and the password maker.

Set up to start the FTP server at bootup

Add this script as /etc/init.d/ftpserver

#!/bin/sh
python /home/robot/ftpserver.py

Then make it executable and let the init system know to run it at boot up:

$ sudo chmod +x /etc/init.d/ftpserver
$ sudo update-rc.d ftpserver defaults

Python 3

Set up to use Python3 (this may not be needed with a future release of ev3dev?)

Note that Python 3 is pre-installed on the ev3dev distribution but some of the supporting libraries are not, which is what makes these steps necessary.

On EV3:

$ sudo apt-get update
$ sudo apt-get install python3-pil
$ sudo python3 -m easy_install python-ev3dev

Note that “apt-get update” may take a while…

Lua Web Server

One of my large and ever-unfinished projects is a Home Automation system. This is designed as multiple processes where most of them are Lua script engines along with several C++ processes as well that handle low-level interfacing and infrastructure for the system.

This is all running on a RaspberryPi.

As part of this effort, I wrote a simple Lua web server to allow access to the system from a browser or a custom iOS app. Since this Lua web server might be useful to others, I am making it available as a separate component.

It is quite simple, but for serving JSON and HTML, and responding to requests (JSON or otherwise) in a low-demand system like personal home automation, it should be just fine.

Note that this depends on the Lua socket library being present.

Lua Web Server repository on Bitbucket

1802 Emulator

In a previous post I talked about the 1802 Assembler I was playing with when I revisited an ancient 1802-based FIG-Forth implementation.

Since it was a pain to get that code running on real hardware, I also threw together an emulator. This was done in Objective-C for the Mac.

Note that the core emulation code is written in ‘C’ however, so it should be quite portable.

It is fairly quick, although I have not tried to quantify it’s equivalent speed.

It will build under recent Xcode versions such as Xcode 6 or Xcode 7.

Listing files can be loaded and executed. There is application-specific code to treat some of the IO ports as serial input/output to a simple terminal facility.

This is in no way a polished final product, but it is made available on the off chance that someone else might find the code useful or interesting.

https://bitbucket.org/don/cosmac-1802-emulator

iOS Apps – Letting Go

At this point I have three apps on the Apple App Store. Dose Tracker, an app that allows you to track how many doses have been taken and how many remain of perhaps an inhaler.
Grounded, a silly app that lets you keep track of the “grounded” status of your children. And GrieveIt, a serious tool for labor professionals.

All three are in various states of neglect, due to my iOS development focus having been my full-time job as an iOS developer at Mellmo for the past several years.

So for those several years, I’ve been feeling guilty about not maintaining or improving these apps. From a financial standpoint, it makes no sense whatsoever. For inexpensive non-games with no advertising budget, the revenue is minuscule.

After giving this a lot of thought, reading blogs and talking with other developers, I think the time has come to remove most of these apps from the store. The one app that I feel I need to at least maintain is GrieveIt, so that one stays. But the other ones are going to go.

I’ll still feel guilty, but not as guilty as I feel about having outdated apps still for sale. And from a “portfolio” standpoint, those don’t really represent the state of the art, so they probably aren’t fulfilling that function either.

I think that marks the point where I clearly segregate my programming into two groups. Things that I do primarily to make money (i.e. my day job), and things I do for fun (my own iOS apps, etc.).

Yes, I know, GrieveIt falls in the middle, sigh. I’m toying with the idea of making it free, since it does help people do things I believe in. And by free, I mean truly free – no in-app purchases, no advertising. Free.
I’m still thinking about this. If I decide to invest some time to update it and add features, I might keep it in the theoretically money-making state. Or I might not – the whole effort to maintain even a simple “business” is a drag on my time and energy as well. (And if I don’t invest the necessary time and energy, then I pay in guilt, so…)

In summary: time to streamline!

Interesting Websites May-2013

This is a collection of interesting and useful websites for iOS development, as well as some more general technical stuff.


General

Apiary

If you deal with REST APIs, this is some pretty neat stuff:

    http://apiary.io

Quandl

Tons of numerical datasets. Not sure what I’m going to do with this, but I feel like I should come up with something:

    http://www.quandl.com

Xcode Tools

XCode Package Manager

An easy way to manage adding packages that modify and improve Xcode.

    http://mneorr.github.io/Alcatraz/

Snippet Editing

This is a nice tool to let you edit your Xcode snippets.

    http://cocoaholic.com/snippet_edit/

Appledoc

If you are looking to document some Objective-C classes or frameworks you’ve created, this is a very easy way to generate documentation that looks like Apple’s. This tool can also create documents that will integrate nicely with Xcode as well.

    http://gentlebytes.com/appledoc/

Objective-C

Objective-C Features

Wondering when a certain feature was available in Objective-C? Wonder no more!

    http://developer.apple.com/library/ios/#releasenotes/ObjectiveC/ObjCAvailabilityIndex/index.html

Classes, Frameworks, and Libraries

Networking

This makes networking so much easier it has to be see to be believed. There are many large and serious iOS apps and products that make use of this class!

    https://github.com/AFNetworking/AFNetworking

Core Data

    https://github.com/magicalpanda/MagicalRecord

Handy Classes

This is a collection of very handy-looking classes for iOS development.

    http://sstoolk.it

Sliding side-panels

    https://github.com/ktatroe/sidepanel-ios

JKFiltering

Filter arrays with blocks

    https://github.com/jklaiho/JKLFiltering

Numeric Entry

    https://github.com/benzado/HSNumericField

Xcode Build Version Script Revisited

On my first release build (versus an Ad Hoc one for TestFlight), I discovered that Apple apparently doesn’t like such a lengthy build version number for releases. (ok, it is rather long…)

An updated version of the script is now available via BitBucket at XCode Tools.

There are a couple other older, undocumented, and essentially experimental tools in that repository as well. Should I revisit and clean up those as well, I’ll add them to a future blog post.

XCode Build Versions

XCode Build Versions

So, one of the things I found myself wanting to do is automatically bump the version of my application. One good reason to do this is when using the amazing TestFlight to distribute beta builds.

Now, we actually have two “versions” to play with. The first is the actual “Version”, and the second is the “Build”. For my purposes, the Build is what I want to change with each release.

This could be an incrementing number, but I’d really like it to be more useful than that. I use Mercurial as my SCM, so what I decided I wanted was to make the Build “number” be the Mercurial changeset identifier. That way (assuming I’ve done a check in of the changes, as I should) the build number will always be different, and meaningful.

What follows is the result of inspiration gained from looking at solutions found by other developers, in particular from the blogs of Bill Woody and Daniel Jalkut.

Script Summary

This script read the changeset ID from Mercurial and writes it to the app’s plist as the build version.

In addition, this script checks to see if we are doing anything other than a “Debug” configuration build with uncommitted source changes, and if so it generates a build error.
(This feature can be customized or disabled as needed)

Script Details

This script could probably be done as a shell script, and certainly could be done using Perl or Ruby. Since Python is my go-to scripting language, that’s what I used.

First, we use the Mercurial command-line program (hg) to get the changeset ID.

We then check to see if the ID ends in a plus-sign. If it does we look at what build configurations are allowed to be run with uncommitted files. Typically, the Debug configuration is the only one you’d want to do with any changes to the source not checked in. This feature can be configured as needed for your particular build style.

Next, we optionally shorten the build version string. You may want to do this if you want a shorter ID. We could take this fairly lengthy hash and truncate it to wind up with a shorter build version that for all practical purposes would still probably be unique.

The path to the build files is created, and then we use the defaults command to update the build version in the plist.
Note that we do this to the plist after it’s been copied to the build products directory. The “source” plist is not modified by design.

Python Script


#!/usr/bin/python
#
# Obtain the current Mercurial version for the local repository, and write this to the
# build version of the XCode project this script is run from.
# Also can check the build configuration and generate an error if uncommited changes exist
# and (for example) a Release build is being made.
#
# This script expects to be run as a build phase between Link Binary With Libraries and
# Copy Bundle Resources.
#

import os
import os.path
import string
import sys


# Set this to true to cause the IDs to be shortened to "XXXX" or "XXXX+"
shortenID = 0

# Configurations that allow uncommitted changes.
# If a configuration like "Release" is not in this list, the script will generate an error
# if there are changes that haven't been committed when a release build is attempted.
uncommittedAllowed = [ 'Debug' ]
#uncommittedAllowed = [ 'Debug', 'AdHoc', 'Release' ]


fh = os.popen( "/usr/local/bin/hg id -i" )  # Ask Mercurial for the global version ID
buildVer = fh.readline().strip()            # Read it from the process, and strip the newline

if buildVer[-1:] == '+':
    # Plus sign at the end means uncommitted changes

    # Check to see if this should cause a build error.
    config = os.environ['CONFIGURATION']
    if not config in uncommittedAllowed:
        # For this configuration, doing a build with uncommitted changes is an error.
        print "ERROR: Uncommitted changes in the repository while doing %s build!" % config
        sys.exit( 1 )

    # Do we want a shorter ID?
    if shortenID:
        buildVer = buildVer[:2] + buildVer[-3:] # First two chars and the last three chars (plus sign)  
else:
    # Do we want a shorter ID?
    if shortenID:
        buildVer = buildVer[:2] + buildVer[-2:] # First two chars and the last two chars

# Build the path to the Info plist in the build products directory.
infoPath = os.path.join( os.environ['BUILT_PRODUCTS_DIR'], os.environ['WRAPPER_NAME'], "Info" )

print "Setting build version of '%s' to PList %s" % ( buildVer, infoPath )

cmd = "defaults write %s CFBundleVersion %s" % ( infoPath, buildVer )
os.system( cmd )

sys.exit( 0 )   # Success

Installation

This can be embedded in the project, or called from the build phase (as shown in the image).
Regardless, it must be placed in the list of build phases between Link Binary With Libraries and Copy Bundle Resources.

Both ways have their pros and cons. If it’s embedded you can’t “lose” it. If it’s located in a common location and called by all your various XCode projects, it’s easy to fix or modify when necessary.
If it was an integral part of the build, I’d prefer having it embedded, but since it isn’t (the project will build just fine if you take it out of the build phases) I’d rather call it.

This script uses the OSX defaults command, and as of this writing (Sep 2011) the man page for that command warns that it will lose the ability to modify plists in this fashion at some point. When that day comes, I’d much rather fix my script once than edit who knows how many projects.

XCode Environment Variables

Environment Variables

When XCode runs a script, it sets a lot of environment variables into the shell environment that the script executes in.

These are some of the most interesting environment variables available to scripts running from XCode.

CONFIGURATION=Debug
INFOPLIST_FILE=MyCoolApp/MyCoolApp-Info.plist
PRODUCT_NAME=MyCoolApp
PRODUCT_SETTINGS_PATH=/Users/donmeyer/Code/ObjC/MyCoolApp/MyCoolApp/MyCoolApp-Info.plist
PROJECT_DIR=/Users/donmeyer/Code/ObjC/MyCoolApp
PROJECT_FILE_PATH=/Users/donmeyer/Code/ObjC/MyCoolApp/MyCoolApp.xcodeproj
PROJECT_NAME=MyCoolApp
PROJECT=MyCoolApp
SOURCE_ROOT=/Users/donmeyer/Code/ObjC/MyCoolApp
TARGET_NAME=MyCoolApp
TARGETNAME=MyCoolApp

A full list can be generated by running (as a build phase) a script consisting of this line:

set >evars.txt

Coding As Always

shapeimage_27.png

So, coding.  Something I do most days at work, but not every day at home.  With the Apple iPad on the horizon, I think it’s time to do some more at-home coding.  And software design of course.

I also plan to get back to Mac development at some point.  In that vein, here’s a very cogent discussion by Brent Simmons on deciding which OS versions to support.
Email sent to a developer on supporting 10.6 and up

Subversion Migration

shapeimage_23.png

I recently started making the move to using Subversion as my version control system.

Since migrating to the Macintosh, I had not gotten CVS up and running. My old system (in the dark PC days) was to have the CVS repository on my working machine, but on a different physical hard drive. On the new iMac, there was no second hard drive. (I do have Firewire drives, but leaving them connected seems to spoil the beautiful simplicity of the G5 iMac.)

Most everything in my old CVS repository is for the Palm, or some other source code that I don’t need to migrate to the Mac. Switching to Subversion seems to make sense now.

I downloaded the pre-built binaries from Martin Ott’s page. Quite nice of Mr. Ott to make those available.

Changes I made:

Note that the location of the global config files is NOT where the SVN docs seem to indicate. On my system using the Coding Monkeys install, they were at: ~/.subversion/config

Made sure that global ignores were enabled in the config file. Added “build” to cover the build sub-dir in an Xcode project. Need to keep in mind that this might be a problem in other contexts! Might be some better way to accomplish the goal of not having any of the build stuff included…

global-ignores = .o *.lo *.la ## ..rej *.rej .~ ~ .# .DS_Store build

Although links to the executables were created in /usr/local/bin, that directory was not on my default path in the shell.

The solution is to either add that directory to the path, or add the subversion bin directory /opt/subversion/bin.

Some general notes:

Creating the repository. I used the FSFS store instead of the BDB.

>svnadmin create –fs-type fsfs ~/svnrepo

Import a directory.

>svn import -m “New” . file:///users/donmeyer/svnrepo/code/objc/projectname/trunk