Random Howtos

How to increase blog traffic

Sat, 05 Jan 2013 15:54:00 -0500

Trying out a few of those tips. Having problems with “#1 Target Your Content to an Audience Likely to Share”.

There are some tumblr specific tips here

Tutorial: Django on Appengine using Google Cloud SQL - Adaptations for Python 2.5

Tue, 01 Jan 2013 15:53:00 -0500

Google App Engine has two Python runtimes, either 2.7.3 or 2.5.2. You should try to develop for 2.7, as it’s the current default on GAE and all the documentations often only describe this Python version.

Additionally: If you use Python 2.5 you can only use Django up to version 1.3. (At least I couldn’t find out how to get version 1.4 working, use_library('django', '1.3') would fail if setting 1.4

If you’re stuck with Python below 2.7 for any reason, that’s what you need to change following my tutorial in order to get it working for Python 2.5:

If you’re Python version is 2.6, you need to get a 2.5 enviroment: either using virtualenv:
```
virtualenv -p /usr/bin/python2.5
```
or simply change the default python version:
OS X:
```
defaults write com.apple.versioner.python Version 2.5
```
Linux:
```
ln -f /usr/bin/python2.6 /usr/local/bin/python
```
In order to make oauth work you need to install ssl (which comes preinstalled with 2.7):
```
sudo /usr/bin/python2.5 /usr/local/bin/pip install ssl
```

Instead of app.yaml defined in the tutorial you need to put 2 files into your project dir: app.yaml and main.py:
app.yaml (replace appproject with the id of your appspot.com instance):

application: appproject
version: 1
runtime: python
api_version: 1

handlers:
- url: /static/admin
  static_dir: static/admin
  expiration: '0'
- url: /.*
  script: main.py

main.py (normally those main.py snippets in the web set DJANGO_SETTINGS_MODULE to mysite.settings, but in my enviroment it only worked when omitting the project name):

import os
import sys
import logging
 
# Google App Hosting imports.
from google.appengine.ext.webapp import util
from google.appengine.dist import use_library
 
os.environ["DJANGO_SETTINGS_MODULE"] = "settings"
 
use_library('django', '1.3')
 
# Enable info logging by the app (this is separate from appserver's
# logging).
logging.getLogger().setLevel(logging.DEBUG)
 
def log_exception(*args, **kwds):
  logging.exception('Exception in request:')
 
# Force sys.path to have our own directory first, so we can import from it.
sys.path.insert(0, os.path.abspath(os.path.dirname(__file__)))
 
# Force Django to reload its settings.
from django.conf import settings
settings._target = None
 
import django.core.handlers.wsgi
import django.core.signals
import django.db
 
# Log errors.
django.dispatch.Signal.connect(
   django.core.signals.got_request_exception, log_exception)
 
# Unregister the rollback event handler.
django.dispatch.Signal.disconnect(
django.core.signals.got_request_exception,
django.db._rollback_on_exception)
 
def main():    
    # Create a Django application for WSGI.
    application = django.core.handlers.wsgi.WSGIHandler()
 
    # Run the WSGI CGI handler with that application.
    util.run_wsgi_app(application)
 
if __name__ == "__main__":
    main()

That’s it, the rest of the tutorial should work!

Troubleshooting

I get pkg_resources.DistributionNotFound: pip==0.8 when trying to pip install

Run this first:

sudo easy_install --upgrade pip

Tutorial: Django on Appengine using Google Cloud SQL

Sun, 30 Dec 2012 16:47:00 -0500

Google is running an introductory trial for Cloud SQL that runs since Nov 2012 until June 1, 2013. That’s the right hour to test Django on Google App Engine. No need to mess around with their non relational datastore. Just use their MySQL 5.5 in the cloud.

Hence I decided to give it a try and documented all into a very complete tutorial how to start a Django 1.4 project running on “Google App Engine”.

If you run into exceptions, check the troubleshooting part at the bottom

Requirements

Python 2.7. If you can’t use 2.7 for any reason, no worries. Here are the adaptions you need to make for this tutorial to work
OS X or Linux. I’m on OS X 10.8.2, I marked the steps that are OS X specific. For Linux users it’s most probably very easy to adapt those. For Windows you certainly need to do some bigger adaptations.
MySQL and its python bindings (for local development): detailed instructions for OS X here

What about Django < 1.4?

Appengine supports the versions 0.96, 1.2, 1.3 and 1.4. This tutorial should work with all those versions. I only tested 1.3 and 1.4 though. All you need to do with this tutorial is to change 1.4 to 1.3 and keep in mind that the Django directory structure slightly changed in version 1.4 so you need to adapt some of the shell commands.

Create a Google App Engine Instance

Create new instance on appengine.
Your instance will be located in the US unless you’re willing to pay 500$ a month to register for a premium account (Europe hosting is for premium accounts only)

Install Google App Engine (OS X specific)

Download the GoogleAppEngineLauncher-x.x.x.dmg
Open the dmg, move GoogleAppEngineLauncher to Applications
Open GoogleAppEngineLauncher, say yes to the symlinks

Add $PATH and $PYTHONPATH to shell environment: Add these lines to .bash_profile (.bashrc on Linux):

export GAE="/usr/local/google_appengine"
export PYTHONPATH="$PYTHONPATH:$GAE:$GAE/lib/django_1_4"
export PATH=${PATH}:$GAE/lib/django_1_4/django/bin/

Load these settings into the current session with

source ~/.bash_profile

and make the django binaries executable:

chmod a+x $GAE/lib/django_1_4/django/bin/[a-z]*.py

Create the django project

run this in a directory of your choice (I chose ~/python/). This generates a stub django project. Replace mysite with the name of your django project.
```
django-admin.py startproject mysite
```
Switch into mysite (where manage.py resides, only do this if you run Django 1.4):
```
cd mysite
```

create the file mysite/app.yaml with this content (replace appproject with the id of your appspot.com instance):

application: appproject
version: 1
runtime: python27
api_version: 1
threadsafe: true

libraries:
- name: django
  version: "1.4"

builtins:
- django_wsgi: on

handlers:
- url: /static/admin
  static_dir: static/admin
  expiration: '0'

edit mysite/settings.py: Change the value of ROOT_URLCONF from mysite.urls to urls (else you run into an exception in your live instance)
also in settings.py add these 2 lines anywhere at the top:
```
import os
BASE_DIR = os.path.abspath(os.path.dirname(__file__)) + os.sep
```
This is the path prefix you’ll put before all the xyz_ROOT settings later.
run this one level up of your python project directory:
```
dev_appserver.py mysite
```

Congrats! Your local instance now shows this - hopefully :-)

Set up Google Cloud SQL

Register Cloud SQL in the Google API Console (I think you need to add billing, but currently there’s a free plan until June 1, 2013)
create a new instance in the United States (even when you’re located in Europe). The reason is that your Cloud SQL instance needs to be at the same location as your GAE instance. Put your ID of your appspot.com instance to the Authorized Applications.
create a new database using the SQL Prompt:
```
CREATE DATABASE my_database;
```

replace the DATABASES section of your settings.py with the snippet below: (Replace my_project:instance1 with your Cloud SQL Instance id and my_database with your created database name).

import os
if (os.getenv('SERVER_SOFTWARE', '').startswith('Google App Engine') or
    os.getenv('SETTINGS_MODE') == 'prod'):
    # Running on production App Engine, so use a Google Cloud SQL database.
    DATABASES = {
        'default': {
            'ENGINE': 'google.appengine.ext.django.backends.rdbms',
            'INSTANCE': 'my_project:instance1',
            'NAME': 'my_database',
        }
    }
else:
    # Running in development, so use a local MySQL database
    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.mysql',
            'USER': 'root',
            'PASSWORD': '',
            'HOST': 'localhost',
            'NAME': 'my_db',
        }
    }

These settings configure django to use a local MySQL storage for development. That’s very close to the cloud setup, as Google Cloud SQL is powered by Mysql (currently 5.5)
I highly recommend this as on my machine every SQL query took about 1 second when run against Google Cloud SQL.
That comes from the fact as first the SQL queries run over HTTP and second the Cloud SQL Instance runs in California.

To trigger the oauth authorization (stored in ~/.googlesql_oauth2.dat) run this:
```
SETTINGS_MODE='prod' python manage.py syncdb
```

If that last command worked that proves that your Cloud SQL worked so far. Congrats!

Your local MySQL instance is now ready as well. You should check if MySQLdb is installed:

python -c "import MySQLdb"

To test the Django↔MySQL connection run

python mysite/manage.py syncdb

Whenever you want to sync your django models to:
a) the live db: SETTINGS_MODE='prod' python manage.py syncdb
b) the local mysql db: python manage.py syncdb

Deploy your stub app to appspot

Ready to deploy your fresh app to the cloud?
Run this (replace mysite with your project name):

appcfg.py --oauth2 update mysite

After about 1 minute your fresh Django project should run perfectly on http://your-id.appspot.com/

Why not sqlite?
Sqlite would be a lot easier to set up, actually there is nothing to install and no server to start, no passwords, etc.
Apart from the fact that I couldn’t get it working (details see here) it’s certainly not a smart idea to run sqlite locally and MySQL (as used by Cloud SQL) in production, it’s very likely that you’ll run into issues.

Serving static files

Django supports serving static files via django.contrib.staticfiles. However, I didn’t get this to work. And since serving these files directly via GAE is faster anyways, add this to your app.yaml:

- url: /media
  static_dir: media
  expiration: '0'

This assumes your static files are under mysite/media. Your static files now serve under /media/

Enable Admin (optional)

You probably want to enable the admin interface (Steps 2-4 are actually all about getting the static admin files to serve. Full discussion see here.)

Uncomment all admin specific lines in settings.py (in INSTALLED_APPS) and urls.py (header and urlpatterns).
Go sure you don’t miss any of these lines by double checking here, under “Activate the admin site”.

Sync the new models agains live

SETTINGS_MODE='prod' python mysite/manage.py syncdb

and local MySQL

python mysite/manage.py syncdb

in settings.py replace the STATIC_ROOT line with (you defined BASE_DIR above):
```
STATIC_ROOT = BASE_DIR + 'static'
```
To copy all the admin media assets into mysite/static run:
```
python mysite/manage.py collectstatic
```
now, /admin should show your admin site, with CSS.

And now?

If you’re new to Django or if you want to be 100% sure everything works as expected follow the django tutorial
Want to port your Django app to Google App Engine? I’ll likely come up with another article about that.

Troubleshooting

on my development machine the web app is veeery slow

Chances are high that you’re not using the local MySQL. Start dev_appserver.py with the --debug flag and see if it does any RPC calls (SQL queries wrapped into HTTP)

If that isn’t the issue you might want to track performance with appstats.

I wanted to use sqlite instead of mysql as a backend, but I run into ImportError, cannot import name utils

So did I. I don’t know how to solve it. See here for details.

I get an unknown locale exception when running syncdb against MySQL

I personally got ValueError: unknown locale: UTF-8. There is a solution for that

I get a DoesNotExist exception when accessing /admin/

At one point I got this exception: DoesNotExist at /admin/, Site matching query does not exist
Solution is described here

I get an ImportError for mysite.urls

I got the exception No module named mysite.urls before I replaced

ROOT_URLCONF = 'mysite.urls'

with

ROOT_URLCONF = 'urls'

I didn’t really understand why that error occurs.

How to install mysql and python bindings on OS X

Sun, 30 Dec 2012 08:45:00 -0500

For my upcoming post “how to create a Django project on Google App Engine” I needed to install a MySQL instance and configure python to do so. MySQL is a little beast, and it is not so straight forward to install it on OS X…

Install mysql. Find good instructions here. I also did the optional step to install the MySQL startup scripts and preference pane. I know I’ll forget how to start the mysql server manually after 2 months, so the entry in System Preferences is handy.

create your database:

mysql -u root -e "CREATE DATABASE my_db;"

Optionally set a password for root (I left it blank as my local mysql server only serves to localhost):

      $ mysql -u root
      mysql> use mysql;
      mysql> update user set password=PASSWORD("NEWPASSWORD") where User='root';
      mysql> flush privileges;
      mysql> quit

Install MySQLdb (the mysql python binding):
```
sudo pip install MySQL-python
```
If you haven’t yet installed pip:
```
sudo easy_install pip
```
Test it:
```
python -c "import MySQLdb"
```
should output no error.

Troubleshooting

Library not loaded: libmysqlclient.18.dylib, Reason: image not found. Solution:
```
sudo ln -s /usr/local/mysql/lib/libmysqlclient.18.dylib /usr/lib/libmysqlclient.18.dylib
```
as described here
IOError: [Errno 13] file not accessible: '/Library/Python/2.7/site-packages/MySQL_python-1.2.3-py2.7-macosx-10.8-intel.egg'. I ran into this problem when building MySQL-python myself.
```
pip install MySQL-python
```
solved this problem
on pip install MySQL-python I got EnvironmentError: mysql_config not found
Before running pip, run this
```
export PATH=$PATH:/usr/local/mysql/bin
```

How to migrate your wordpress to tumblr. Including images and comments.

Fri, 14 Dec 2012 05:37:00 -0500

So I’ve decided to move my wordpress blogs to tumblr. Although apparently TechCrunch thinks that’s a bad idea. And although Moritz Adler would kill me for that. (Although: He doesn’t have a personal blog and hence has no licence to kill me). Anyway. With tumblr I don’t need to host a blog software myself. And I don’t end up having my blog hacked and then seeing my blog being displayed as a malware site in Chrome/Firefox (happened to me twice). And then with tumblr I create new blogs with subdomains within minutes. Cool stuff. Hail to the cloud, baby!

So here you go: A complete guide how to fully migrate your wordpress blog to tumblr. Including comments and pictures. And still supporting your old url scheme.

Update: I ran into a tool that claims to do a lot for you: import2.com/tumblr. It doesn’t migrate images and 302 redirects. Not sure about comments migration. And it costs 24$. Still, if you can leave out some of the steps below that’d be worth the money. Comments of the author on quora

Before you start

Before you start to actually move your blog, you need to consider a few things:

Where do you move your DNS to?

If you have a wordpress webhost, then this webhost most probably also does DNS for you. You need to replace that by a third party solution. I think these are good services:

he.net. Free service. No strings attached. The one I’ve chosen. The interface is nice and very easy to add new CNAMES, MX records, etc. The uptime was reported to be not so good, but I don’t really care about uptime of my blogs. To host your tumblr blog on your domain, you add a CNAME to domains.tumblr.com and then configure your tumblr blog to listen to that domain. Very simple.
Amazon Route 53: They charge you $0.50 per hosted zone per month. That’s a fair price and probably has a better uptime then he.net

What do you want to do with your images?

At default they’re all located under www.yourolddomain.com/wp-content/img1.jpg. To completely get rid of your old web host you need to move those to a different image hoster. I don’t advice you to upload it to tumblr because if in future you want to move away from tumblr you run into the same problem again.

Would you like to keep your comments?

Do you have comments at all? Tumblr doesn’t support comments by itself. Most themes have disqus support. Moving comments to disqus is no big deal, but still it’s some work, so you may decide to just not migrate comments.

Are you keen to not break your old blog urls?

Wordpress’ url scheme generally is e.g. code.pui.ch/2007/01/05/print-hello-world/.
That same post ends up at this tumblr url: howto.pui.ch/post/37471154429/print-hello-world. Note that the last part of the url is optional, i.e. howto.pui.ch/post/37471154429 works as well.
If you care about incoming links to your blog not to break and if you care about your google ranking (I guess 302 redirects inherit the google ranking), there are two possibilities:

Stay on the same domain, handle the redirects in tumblr (tumblr supports that with the pages’ type “redirect”)
Move to a different domain and put up e.g. redirect permanent in a .htaccess file on your web server

A rough outline of what you’re up to

At a glance, that what you’ll do:

Upload your images to a different hoster (if you want to get rid of your old webhost)
Extract all blog posts+comment from wordpress
Fix the export.xml: Replace images, more-tags and fix some additional stuff
Migrate your wordpress blog to blogger
Migrate your blogger blog to tumblr
Install http redirects (on old webhost or on tumblr)
Migrate your comments to disqus
Clean up blog posts (might be a biggie if you’re a perfectionist)

Alright, let’s specify those 8 steps.

Step 1: Upload your images to a different hoster

If you’re ok with keeping your old webhost you can skip this point. Easiest thing to do would be to copy your wp-content directory one to one to a different hoster so www.yourolddomain.com/wp-content/img1.jpg turns into www.imagehoster.com/my_user_name/img1.jpg.

I don’t have experience with image hosting providers so I just uploaded my images to picasa, but that meant I needed to update every single image in all my blog posts to the new image url of picasa. That was quite a pain. I couldn’t find a image hoster yet who meets the criterias above. Photobucket doesn’t, Dropbox doesn’t, Google Drive doesn’t. Maybe Amazon S3

Step 2: Extract your blog posts+comment from wordpress

In your wordpress admin go to Tools → Export. On my blog that was code.pui.ch/wp-admin/export.php.
Choose All content, Download Export File.

Step 3: Fix the export.xml

You have just downloaded an xml file, in my case the name was coderandom.wordpress.2012-12-13.xml. Open that file with your favourite text editor. Now you need to do a few things before you can go on:

Replace all <!—more—> by [[MORE]]. The uppercase actually matters. [[MORE]] is the divider that tumblr actually understands as the place where you want your excerpt to stop in the blog overview view.
Replace all images by the new urls you got by uploading the images to the image hoster in step 1. It’s much easier to do this at this stage than to replace the images once you’ve migrated your blog to tumblr.
You may have more wordpress plugins you used in your posts. I used [python]...[/python] to syntax highlight my python markup. I’ve moved to google code prettify which needs <pre class="prettyprint">...</pre> as a syntax. So I needed to replace all occurrences by the new markup. Obviously, regex is your friend at this stage.

Step 4: Migrate your wordpress blog to blogger

Unfortunately there’s no direct way to directly import this xml file into tumblr. Instead, that’s what you need to do:

Convert your xml file on this website to a file fit for importing into blogger.
This doesn’t work for files bigger than 1MB. If that’s the case then you can either convert the file on your own machine using this sourcecode on google code. Or you can follow the instructions under step 3 on this blog post.
Create a new blog on blogger.com.
In blogger navigate to your new blog and do settings → other settings → import blog
Publish all blog posts: Posts → All → Select all → Publish

If you have difficulties in this step you might try a different solution (didn’t try any of these):

Step 5: Migrate your blogger blog to tumblr

Go to bloggertotumblr.com and enter your url for your newly created blogger and tumblr blogs. The conversion is very straight forward.
Delete your blogger blog
Enjoy. Your blog posts are now on tumblr. Still missing: Support of the old url scheme and comments

Step 6: Install http redirects (on old webhost or on tumblr)

In the section before you start I asked you to decide if you’re ok to break your old urls. If you don’t care, then skip this step. Although: if you care about keeping your comments then you still might to do this step, as it makes migrating comments to disqus a lot easier

If you do care, then:

If you keep the domain of your blog: You need to install tumblr redirect pages. See below.
If you changed the domain of your blog: You need to install redirects on your former wordpress webhost. I describe how this is done via .htaccess config below

Install tumblr redirect pages

go to your new tumblr blog
click customize top right
add a page (left column)
instead of “Standard Layout” choose “Redirect” and add a posts’ old url on top, and the new url on bottom. Repeat this for every blog posts (yeah, lots of work here)

Install .htaccess on your old webhost

IMO that’s a bit simpler than installing a tumblr redirect for every blog post. Still it’s a lot of work since you need to come up with a map of old_url → new_url for every blog post. If your webhost supports .htaccess then go for this method. Most probably you have such a file already for your wordpress installation.

An example of a .htaccess file:

  RewriteEngine On
  Redirect permanent /page/2/ http://howto.pui.ch/page/2
  Redirect permanent /feed/ http://howto.pui.ch/rss
  RewriteRule ^$ http://howto.pui.ch/ [R=301,L]
  Redirect permanent /2007/02/25/add-bandwidth-to-a-file-download-in-python/ http://howto.pui.ch/post/37471156141/add-bandwidth-to-a-file-download-in-python
  Redirect permanent /2007/06/08/python-find-out-cpu-time-of-a-certain-process/ http://howto.pui.ch/post/37471156554/python-find-out-cpu-time-of-a-certain-process
  Redirect permanent /2007/07/23/python-sort-a-list-of-dicts-by-dict-key/ http://howto.pui.ch/post/37471157116/python-sort-a-list-of-dicts-by-dict-key
  Redirect permanent /2009/12/30/dealing-with-mysql-backend-does-not-support-timezone-aware-datetimes/ http://howto.pui.ch/post/37471158729/dealing-with-mysql-backend-does-not-support
  Redirect permanent /2011/01/19/python-easy-way-to-show-progress/ http://howto.pui.ch/post/37471159741/python-easy-way-to-show-progress
  Redirect permanent /2012/07/17/how-to-detect-a-files-character-encoding/ http://howto.pui.ch/post/37471161169/how-to-detect-a-files-character-encoding
  Redirect permanent /2010/04/04/python-display-refreshing-status-like-top/ http://howto.pui.ch/post/37471159398/python-display-refreshing-status-like-top
  Redirect permanent /2011/11/03/how-to-switch-gnu-screen-windows-in-iterm2-via-keyboard-shortcuts/ http://howto.pui.ch/404
  Redirect permanent /2007/01/19/sched-20-pizza-is-ready/ http://howto.pui.ch/post/37471155110/sched-20-pizza-is-ready

Note the first 3 lines, you will need the same for your redirects. Note the special syntax of line 3. That is important to not redirect a non existing url to the new domain.

Wait! I just moved my blog away from my old webhost, and now I need to keep it to have http redirects in place?

Uh, yes. Alternatively, you can move to a free hoster like google apps engine and e.g. use this to redirect. Or you just wait a few months and wait until all search engines have digested your redirects and kill your old webhost only then (that’s what I’m probably gonna do).

Step 7: Migrate your comments to disqus

Because tumblr doesn’t offer comments by itself you need to migrate your comments to disqus:

Register your new tumblr blog url at disqus
On disqus go to Admin → Tools → Import/Export → Upload WXR. Choose the XML file you downloaded from your wordpress installation (not that got converted for blogger). Upload that
On the same page go to Migrate Threads. Choose “Redirect crawler” if you installed the redirects in step 6. Otherwise you need to use the “Upload a URL map” option

install disqus manually into your theme

Voilà! After a few minutes your comments should appear in your new tumblr blog.

Step 8: Clean up your tumblr posts

Might be that the html markup of my wordpress posts were so bad, but I needed to fix a lot of spacing between paragraphs. I also had some CSS tweaks to right align the images which I needed to fix.

Lastly, blogger adds these paragraphs to the end of every blog post:

  <div class="blogger-post-footer">
    <img alt="" height="1" src="https://blogger.googleusercontent.com/tracker/290349385069691835-5946149615494229188?l=coderandomm.blogspot.com" width="1">
  </div>

If you are a perfectionist you may want to remove this markup from every blog post.

A word of caution: You can spend a lot of time at this step if you overdo it.

Thanks

Thanks to MG Siegler (one of my favourite bloggers, although he is an Apple fanboy and I am a google disciple): He showed me that you can perfectly write long blog posts on tumblr (that linked article is actually a must read about Android vs. iPhone)
Julio Angel Ortiz, who’s article served as the base for this howto.
TextMate for making html editing so easy. Just found out that it’s actually a pretty decent HTML editor. I’ll write my blog posts in here and only then past them into the tiny tumblr HTML editor popup.

How to fix Jambox' "static noise and no bluetooth sound" problem(includes soldering)

Sat, 21 Jul 2012 17:07:00 -0400

Jambox is a pretty cool device: The sound quality is very good, it is small, it has a battery. I liked it. Until it broke. It just didn’t play music over bluetooth any more but instead uttered static noise. This seems to be a quite severe production problem as many many many people have devices with the exact same problem. So going down the “Jambox please replace my device” way didn’t sound promising to me. The possibility that the replacement device is broken as well is just too high.

The problem lies in the aux in port. The device thinks there’s an aux cable plugged in and outputs the signal from the aux input when in fact it should play the bluetooth sound.

In my last post I described the different easy-to-do solutions to this problem, but today I needed to implement the soldering solution because nothing else worked. Implementing this solution means tricking the Jambox to think there’s no aux cable plugged in.

Update: My initial soldering was working, but after the first recharge the fix stopped working. Might be that the soldering melted and if you are a solder pro this might work for you. Anyway. I personally ended up putting a screw into the aux port.

What do you need?

a star shaped screw driver (aka torx)
Soldering equipment
being ok that the aux cable will never work again
desparation to get this machine working again (I’m not sure how safe this is, being not used to that kind of stuff), and of course it will void your warranty

I personally hate soldering, I’m not good at it, but still I managed to get my Jambox working agagin so I thought I’d share with the world:

How to disassemble

There’s a good video describing how it should be done (I did it just with the screwdriver. No gloves and no other equipment.)
You need to follow the video until 5:00

What to solder?

The two metal contacts you need to bridge

Update: David Choi has made a video that shows very well which two contacts to solder. He claims that with good soldering skills you can achieve that both bluetooth and aux will still work.
It’s pretty simple: Follow the aux port and where the end of the aux jack would be there are 2 metal contacts which need to be soldered together. Turn on the jambox and try to push the upper metal contact so the two metal pieces touch and you’ll hear the bluetooth sound. Hearing the music should give you enough motivation now to go on. Now:

Turn off the Jambox
Try to solder (it is tricky because the spot is hard to reach)
Try if it has worked. If it worked, wait some time. Shake the bluetooth device to really go sure the soldering joint is good (first time I was so happy that it worked, I assembled the device again and then it didn’t work because the soldering joint was flaky)

How to assemble

I needed to bend the two metal pieces
at the end of the grid to be able to assemble it

Again follow the video starting from 10:40.
I found the assembling the hardest part. In the end I needed to bend the small metal pieces of the grid to be able to put it back together (although I didn’t follow the video so you might be more successful).

Happy listening

My Jambox is now up and running again since 24h. I’m very very happy that this little thing did the trick.

How to detect a files character encoding

Tue, 17 Jul 2012 03:59:00 -0400

iconv is alright - except it’s sometimes hard to know in which encoding the file is I just got.

There is enca which promises to work, but I couldn’t get it working. I was looking for a simple tool that just outputs the encoding without the need of any parameters

Now the python module chardet comes to the rescue. Once you have git forked you can write this boilerplate python script “detect.py”:

#!/usr/bin/python
import sys, chardet
a = open(sys.argv[1]).read()
print chardet.detect(a)

after
chmod a+x detect.py
you can write:
./detect.py my_strange_file.txt

How to reset Jambox when bluetooth completely stopped working

Sat, 14 Jul 2012 13:26:00 -0400

I bought a Jambox about half a year ago. Sound wise it is great, but apparently it is not very stable, especially after recharging it falls into some state where it only utters static noise. In this state it still plays music over the aux cable, but not any more over bluetooth.

Apparently the issue is the aux port which has two contacts that should touch each other when there is no aux cable inserted. But, because of manufacture problems these two contacts don’t touch on some device, even though there is no aux cable inserted. So: The jambox thinks there is an aux cable, but there is none. That’s why it utters static noise

So, because I have a device with such problems I needed to hard reset my jambox half a dozen. That’s why I thought I finally write it down.

Soft reset: Holding down the circle button (talk button) and plugging it into the charger and then releasing. You will see the red flash
Unpair / Pair again; being deleting it from every device, restarting all of them and pairing once more
Hard reset: Turn off bluetooth on all devices in reach, then turn on the jambox and press the circle button six times, when you see the light flash red, press the circle once more and hold it and it will go into pairing mode
Update the software to >=2.1 using mytalk.jawbone.com
Insert an aux cable and removing it again (worked a few times for me)
Insert a small screw driver (this is scary, more detailed instructions here). This connects the two metal contacts which should touch each other to make the Jambox believe there is no AUX cable inserted
If the screw driver thing is working for you, you might consider putting a screw into the aux port:
If nothing of the above works, you can solder a bypass of the aux port (which is what I finally did)

I don’t recommend buying a Jambox to anyone, I completely agree with this review on Amazon

keyboard shortcuts in iTerm2 on OS X

Mon, 25 Jun 2012 09:46:00 -0400

I love having my terminal completely on the server, so I can connect from everywhere and just resume with all my windows.

That’s why I always work on tmux and my iTerm just has one tab.

That’s why I remapped e.g. cmd-t to “new tmux tab” and cmd-w to “close tmux tab”.

To achieve this, in tmux’ preferences in keys you need to add global shortcut keys. I’m using ctrl-z as prefix, if you use ctrl-a use 0x01 instead. These are the combinations for ctrl-z (List items are: Key combinations - Hex Codes - Comment):

⌘0 - 0x1a 0x22 - switch to window 0. Other window suffixes (replace 22 by that): 1=31,2=32,3=33,4=34,5=35,6=36,7=37,8=38,9=39
⌘r - 0x1a 0x2c 0x08 0x08 0x08 0x08 0x08 - rename current window (0x08 sends backspace)
⌘t - 0x1a 0x63 - new “tab” (=new tmux window)
⌘w - 0x04 - sends ctrl-d
⌥⌘← - 0x1a 0x1b 0x5b 0x44 - jump to plane to the left (ctrl-z ” or ctrl-z % to split windows into panes in tmux). Replace 0x44 by 0x41 (up), 0x42 (down), 0x43 (right)
⌘↑ - 0x1a 0x5b - jump to tmux history

Use showkey --ascii on the shell to inspect the hex keys of key strokes.

Python: easy way to show progress

Wed, 19 Jan 2011 06:53:00 -0500

I use this quite often to indicate how long a loop takes:

for i, entry in enumerate(entries):
  # do something
  print "\r%s/%s" % (i, len(entries)),
  sys.stdout.flush()

It shows
1/13
2/13
…
on the same line (similar to a progress bar)

Python: display refreshing status (like top)

Sun, 04 Apr 2010 18:24:00 -0400

In scripts I often want to display some status information (e.g. progress), this can be achieved with e.g.:

print "progress: %i %%\r" % i,
sys.stdout.flush()

but this just works for one-liners. I wanted to have what top or less do: open a new “window” and being able to write everywhere in the window, not just on the last line.

Found out that curses does exactly that - but I didn’t find a stripped-to-the-bare-necessities-example, so here you are:

import curses, time
from datetime import datetime
w = curses.initscr()
try:
  while True:
  w.erase()
  w.addstr("some status..\ncurrent time\n%s" % datetime.now())
  w.refresh()
  time.sleep(1)
finally:
  curses.endwin()

Python: Print list of dicts as ascii table

Sat, 27 Feb 2010 06:55:00 -0500

Sometimes I want to print list of dicts as an ascii table, like this:

  | Programming Language | Language Type | Years of Experience |
  +----------------------+---------------+---------------------+
  | python               | script        |                    4 |
  | php                  | script        |                    5 |
  | java                 | compiled      |                   11 |
  | assember             | compiled      |                   15 |

I searched on Google - but without luck.

That’s what I came up with - it’s not particularly nice but it does the job:

  def table_print(data, title_row):
    """
    data: list of dicts,
    title_row: e.g. [('name', 'Programming Language'), ('type', 'Language Type')]
    """
    max_widths = {}
    data_copy = [dict(title_row)] + list(data)
    for col in data_copy[0].keys():
      max_widths[col] = max([len(str(row[col])) for row in data_copy])
    cols_order = [tup[0] for tup in title_row]

    def custom_just(col, value):
      if type(value) == int:
        return str(value).rjust(max_widths[col])
      else:
        return value.ljust(max_widths[col])

    for row in data_copy:
      row_str = " | ".join([custom_just(col, row[col]) for col in cols_order])
      print "| %s |" % row_str
      if data_copy.index(row) == 0:
        underline = "-+-".join(['-' * max_widths[col] for col in cols_order])
        print '+-%s-+' % underline

Use it like that:

  data = [dict(name='python', type='script', years_experience=4),
    dict(name='php', type='script', years_experience=5),
    dict(name='java', type='compiled', years_experience=11),
    dict(name='assember', type='compiled', years_experience=15)
    ]
  titles = [('name', 'Programming Language'),
    ('type', 'Language Type'),
    ('years_experience', 'Years of Experience')]
  table_print(data, titles)

It will produce the table printed above. It’s not fancy - the only ‘smart’ thing it does is right-adjusting integers, strings are left-adjusted.

P.S. no, I don’t have 15 years of experience of Assembler - I just know it since 15 years - it’s one of the first programming languages I learned - and I even wrote a text editor with it - then I learned that’s probably not the best language to write an editor :-)

dealing with "MySQL backend does not support timezone-aware datetimes"

Wed, 30 Dec 2009 18:27:00 -0500

When fetching events from an iCal feed and saving this into a database I got

MySQL backend does not support timezone-aware datetimes

This did the trick for me: Install pytz (download here)

import pytz
that_datetime_in_utc.astimezone(pytz.timezone('Europe/Zurich')).replace(tzinfo=None)

Caution: I don’t really understand what I’m writing here - it feels like those posters in PHP forums who explain ‘how to …’ and then trying to explain something they have no clue of, but well.. :-)

Fetch publicly available google calendar data with python

Tue, 29 Dec 2009 16:31:00 -0500

I tried accessing the google data api with python - that api seems either overly complicated or just not suited for just grabbing events from a public google calendar.

This worked for me:

find out ical address (subscribe to the calendar, other calendars->arrow down->calendar settings)
install icalendar

Then this is possible:

      
from icalendar import Calendar
import urllib
ics = urllib.urlopen('http://www.google.com/calendar/ical/fchppllvcaupb6fgguigobkfj4@group.calendar.google.com/public/basic.ics').read()
ical=Calendar.from_string(ics)
for vevent in ical.subcomponents:
 if vevent.name != "VEVENT":
  continue
 title = str(vevent.get('SUMMARY'))
 description = str(vevent.get('DESCRIPTION'))
 location = str(vevent.get('LOCATION'))
 start = vevent.get('DTSTART').dt      # a datetime
 end = vevent.get('DTEND').dt        # a datetime

Send javascript errors by mail

Sat, 06 Oct 2007 09:33:00 -0400

I’m running a Django-powered site for a closed user group and added a bit of JavaScript magic here and there (mainly Prototype and Tooltip).

Now Django sends me a mail whenever a 404 or 500 error occurs. But when one of my users encounters a JavaScript-Error, I’m not informed. I thought anyone in the web has solved this problem but didn’t find anything, so here’s my take: Just send any error using Ajax (here: using Prototypes Ajax abstraction) to the server

onerror = Extranet.mailError;
function mailError(msg, url, line) {
 var postBody = 'url=' + url + '&line=' + line + '&message=' + escape(msg) + '&useragent=' + escape(navigator.userAgent) + '&user=' + escape(user_name);
 var myAjax = new Ajax.Request('/api/jserror/', {method: 'post', postBody: postBody});
}

user_name is a JavaScript variable holding the Django username (so I know whom I can inform when the error is fixed).

On the server side, I just send me mails containing the JavaScript error message, the username and the user agent:

def jserror(request):
 from django.core.mail import mail_admins
 omit_messages = ['pointerobj is not defined', 'tipobj is not defined', 'ns6 is not defined', 'enabletip is not defined']
 if request.POST.get('message', '') not in omit_messages:
  message = """url: %s (%s)
%s
user-agent: %s
username: %s
""" % (request.POST.get('url', ''), request.POST.get('line', ''), request.POST.get('message', ''), request.POST.get('useragent', ''), request.POST.get('user', ''))
  mail_admins("javascript error", urldecode(message))
 return HttpResponse()

Yeah, that’s all very trivial but I wonder what other solutions exist for this problem…

Django: Serve big files via fcgid

Wed, 03 Oct 2007 04:14:00 -0400

I’ve got a django project running which requires you to login to access files.
That means that I have to serve the files via python, like this:

  @login_required
  def download(request, filename):
    # ... some code specific to my site ...
    response = HttpResponse(mimetype=postUpload.mimetype)
    response['Content-Disposition'] = "attachment; filename=" + original_filename
    response['Content-Length'] = os.path.getsize(filename_path)
    response.write(open(filename_path).read())
    return response

The problem: If the download of a file exceeded 5 minutes (big files and/or low bandwidth) the download was canceled on the server side by a timeout. This Apache configuration for mod_fcgid solved the problem (see mod_fcgid documentation for BusyTimeout)

<IfModule mod_fcgid.c>
 BusyTimeout 1200
</IfModule>

The problem was that the apache module scanned every minute for processes that run for more than BusyTimeout seconds. These processes are potentially in bad health (infinite loop et al.) and have to be killed. Not so with my processes (since I know what I’m doing..). The setting of the busy timeout to 1200 seconds now lets my processes run for a maximum of one hour.

As this setting can’t by overwritten in a htaccess file by default I needed to bug my web hosting provider with the request, which was handled in 24 hours, so thanks for that one!

PS: If you know of another way how to serve protected static files via a single sign on (no HTTP basic auth), please let me know.

Python: Sort a list of dicts by dict-key

Mon, 23 Jul 2007 10:52:00 -0400

I always forget that one and end up searching for half an hour:

  >>> l = [ dict(a=1,b=2,c=3),
  ... dict(a=2,b=2,c=2),
  ... dict(a=3,b=2,c=1)]
  >>> l.sort(key=operator.itemgetter('c'))
  >>> l
  [{'a': 3, 'c': 1, 'b': 2}, {'a': 2, 'c': 2, 'b': 2}, {'a': 1, 'c': 3, 'b': 2}]
  >>> l.sort(key=operator.itemgetter('a'))
  >>> l
  [{'a': 1, 'c': 3, 'b': 2}, {'a': 2, 'c': 2, 'b': 2}, {'a': 3, 'c': 1, 'b': 2}]

Update: This is a more elegant way to do the same:

>>> l = [ dict(a=1,b=2,c=3),
... dict(a=2,b=2,c=2),
... dict(a=3,b=2,c=1)]
>>> sorted(l, key=lambda k: k['c'])
[{'a': 3, 'b': 2, 'c': 1}, {'a': 2, 'b': 2, 'c': 2}, {'a': 1, 'b': 2, 'c': 3}]

Python: Find out cpu time of a certain process

Fri, 08 Jun 2007 06:52:00 -0400

To find out how many percentage a certain process uses the cpu:

  import os, time

  # find out the pid by username.
  # "-o pid h" omits the header and just prints the pid
  pid = os.popen('ps -U my_user_name -o pid h').read().strip()

  # 14th column is utime, 15th column is stime:
  # The time the process has been scheduled in user/kernel mode
  # The time value is in jiffies. One jiffie is appox 1/100 second
  # see man proc for more info
  stat = os.popen('cat /proc/%s/stat' % pid).read().strip()
  cpu_time1=int(stat.split()[14]) + int(stat.split()[15])
  time1=time.time()

  time.sleep(1)
  stat = os.popen('cat /proc/%s/stat' % pid).read().strip()
  cpu_time2=int(stat.split()[14]) + int(stat.split()[15])
  time2=time.time()

  print str(float(cpu_time2 - cpu_time1) / (time2 - time1)) + "%"

I don’t know though if the number is accurate.
What is “cpu time” anyway? It’s the time the process is running (using the cpu for 100%) divided by the time the process is laid asleep by the scheduler.
Then, jiffies seem to be not a safe number for time measurements.
But for relative measurements it should do the trick.

add bandwidth to a file download in python

Sat, 24 Feb 2007 21:51:00 -0500

Another short code snippet I came up when writing the module download script

The following class keeps a download under a certain ratio and prints live download stats (KB/s).

  class BandwidthFile:
    """
    limits the file download to a certain amount of bytes per second and prints download stats
    """
    def __init__(self, filename, filesize, bytes_per_second = 20000, mode = "wb"):
      self.fp = open(filename, "wb")
      self.bytecounter = 0
      self.started = datetime.datetime.now()
      self.bytes_per_second = bytes_per_second
      self.filesize = filesize
      self.filename = filename

    def __del__(self):
      self.fp.close()

    def write(self, bytestring):
      self.bytecounter += len(bytestring)
      seconds_since_started = (datetime.datetime.now() - self.started).seconds + 1
      while (self.bytecounter / seconds_since_started)> self.bytes_per_second:
        seconds_since_started = (datetime.datetime.now() - self.started).seconds + 1
        time.sleep(0.1)
      self.fp.write(bytestring)
      sys.stdout.write("\rdownloading %s (%i/%i KB, %i KB/s)" % (self.filename, (self.bytecounter / 1024), (int(self.filesize) / 1024), (self.bytecounter /seconds_since_started / 1024)))
      sys.stdout.flush()

To use the class with ftp:

  from ftplib import FTP
  ftp = FTP("ftp.modland.com")
  filesize = ftp.size(remote_filename)
  f = BandwidthFile(local_filename, ftp_filesize)
  ftp.retrbinary('RETR '+remote_filename, f.write)

The status line looks like this:

  downloading /home/phred/modules/Skaven/beyond the network.it (95/4411 KB, 19 KB/s)

If you have suggestions how to enhance the code snippet feel free to leave a comment!

set timeout for a shell command in python

Sun, 18 Feb 2007 19:04:00 -0500

I wanted to run a shell command in python without knowing if the shell command is going to exit within reasonable time (adplay that was, sometimes it simply hangs).

Update: the “task” module of Rob Hooft seems to solve this exact problem. At the time I wrote this, the python.net website was down. I leave my solution here just for archive purpose.

  def timeout_command(command, timeout):
    """call shell-command and either return its output or kill it
    if it doesn't normally exit within timeout seconds and return None"""
    import subprocess, datetime, os, time, signal
    start = datetime.datetime.now()
    process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    while process.poll() is None:
      time.sleep(0.1)
      now = datetime.datetime.now()
      if (now - start).seconds> timeout:
        os.kill(process.pid, signal.SIGKILL)
        os.waitpid(-1, os.WNOHANG)
        return None
    return process.stdout.read()

Note especially lines 6, 11 and 12.
Usage:

  >>> output = timeout_command(["sleep", "10"], 2)
  None
  >>> output = timeout_command(["sleep", "1"], 2)

The process can be killed when it has run for too long (the os.waitpid waits for the kill to end and avoids defunct-processes) and furthermore the Popen’ed process’ printed is caught and returned if it doesn’t timeout. However, subprocess.Popen is called with a list as argument. That means, that the command isn’t passed to a shell and furthermore you can just call one command with options, nothing more.

Random Howtos

How to increase blog traffic

Tutorial: Django on Appengine using Google Cloud SQL - Adaptations for Python 2.5

Troubleshooting

Tutorial: Django on Appengine using Google Cloud SQL

Requirements

What about Django < 1.4?

Create a Google App Engine Instance

Install Google App Engine (OS X specific)

Create the django project

Set up Google Cloud SQL

Deploy your stub app to appspot

Serving static files

Enable Admin (optional)

And now?

Troubleshooting

Further reading

How to install mysql and python bindings on OS X

Troubleshooting

How to migrate your wordpress to tumblr. Including images and comments.

Before you start

Where do you move your DNS to?

What do you want to do with your images?

Would you like to keep your comments?

Are you keen to not break your old blog urls?

A rough outline of what you’re up to

Step 1: Upload your images to a different hoster

Step 2: Extract your blog posts+comment from wordpress

Step 3: Fix the export.xml

Step 4: Migrate your wordpress blog to blogger

Step 5: Migrate your blogger blog to tumblr

Step 6: Install http redirects (on old webhost or on tumblr)

Install tumblr redirect pages

Install .htaccess on your old webhost

Wait! I just moved my blog away from my old webhost, and now I need to keep it to have http redirects in place?

Step 7: Migrate your comments to disqus

Step 8: Clean up your tumblr posts

Thanks

How to fix Jambox' "static noise and no bluetooth sound" problem(includes soldering)

How to disassemble

What to solder?

How to assemble

Happy listening

How to detect a files character encoding

How to reset Jambox when bluetooth completely stopped working

keyboard shortcuts in iTerm2 on OS X

Python: easy way to show progress

Python: display refreshing status (like top)

Python: Print list of dicts as ascii table

dealing with "MySQL backend does not support timezone-aware datetimes"

Fetch publicly available google calendar data with python

Send javascript errors by mail

Django: Serve big files via fcgid

Python: Sort a list of dicts by dict-key

Python: Find out cpu time of a certain process

add bandwidth to a file download in python

set timeout for a shell command in python