python on Roy Tang

Flask vs Django

hello@roytang.net (Roy Tang) — Fri, 24 Jan 2020 00:00:00 +0000

In a bid to reduce the number of webapps actually running on my server (for resource consumption reasons), I decided to migrate a small Flask app I had and merge into this larger Django app where I have a lot of my personal data tracking stuff. The Flask app was small enough, mostly containing backend support for this blog (like search and comment submissions) and some Twitter things. The migration was straightforward, taking around half a day, most of that was wrangling with Twitter API rate limits.

It got me thinking, why did I build this using Flask in the first place? And that led to the more general question: when do I prefer using Flask over Django?

Personal experience: I’m much more familiar with Django than Flask, having started using it back in 2008. Professionally, I’ve also worked on more Django projects, compared to only one Flask project that actually made it to production.

The usual quick comparison is that Flask is the more lightweight web framework, suitable to smaller apps like microservices perhaps; while Django is the big monolithic web framework with all the bells and whistles already available. In practice this means Django already provides a lot of functionalities out of the box, while with Flask you have to install and figure out extensions for even the most common tasks like database management. A negative for Flask, but partly due to my inexperience: if you were already well-versed in the use of those extensions, I imagine getting everything together and setting them up will be easier.

If there is need for a traditional RDBMS, I will probably use Django. I’m not super familiar with SQLAlchemy, the usual recommended ORM for Flask, so that’s a big factor. That being said, if I needed to work with something like a NoSQL database or some other nontraditional backend, it might be better to consider Flask since Django isn’t super flexible in that record - most of the pluses are oriented towards RDBMS-backed models.

Admin: The built-in Django admin is a big plus for me; it means that for things like simple record CRUD, if usability and presentation isn’t a big concern, I can just use the Django admin as my backend without needing to write maintenance screens. There is an equivalent Flask Admin plugin, but I’ve not tried that either.

API building: I’m already familiar with the Django REST framework, I’m not sure if there’s an equivalent for Flask. The one Flask project we did, we rolled our own API backend.

I find that I tend to favor Flask when building quick proof-of-concept apps that don’t need a database. An example that was part of the app I migrated recently would be this quick Twitter app I wrote about a year ago, to find intersections between follows/followers. It didn’t need a database, only twitter API calls. (I have since merged it into the Django app, so the current version is using Django now.)

In general I prefer Django, but that is probably due to familiarity and experience. I would love to have a full-scale project that uses Flask though! Perhaps in the future, for now all my personal projects are running on Django.

Python: Markov Chains

hello@roytang.net (Roy Tang) — Wed, 28 Aug 2019 00:00:00 +0000

Back when I was still learning Python in 2008, one of the first “fun” scripts I wrote was a text generator using Markov chains. I’d run it against all the chat logs I had with people at work and serve the results from a webserver on my computer. THe results were often amusing and sometimes hilarious.

Since I’ve been going through my old scripts lately, I thought I’d update that script to Python 3 (read: add parentheses around print params and use pathlib) and run it against all the posts on this here site. I added the script to my deploy script for this site (thus further worsening my build times), so it should generate a new page every hour or so. I also added the generated markdown file to gitignore so it never gets saved to the repo. Every output of this script will be fleeting and ephemeral!

You can view the output here. You can view the script source here.

Devnotes: Python Pathlib

hello@roytang.net (Roy Tang) — Tue, 27 Aug 2019 00:00:00 +0000

Ever since I started learning Python back in 2008ish, I’ve been using it as my primary scripting language for various tasks such as processing log files, organizing my own file system, processing stuff on this blog, and so on. A lot of it is basically moving files around. In the days of Python 2, that involved a lot of imports of different libraries like os, shutil and glob. It can become a bit messy with so many imports, and I often can’t remember which import I need for a particular case and end up having to search for the documentation (or stackoverflow, let’s not kid ourselves here).

With Python 3, a new cleaner option is available to replace all of the above libraries: Pathlib, introduced in Python 3.4, provides an object-oriented way of doing file operations, replacing many of the most common uses I had for the libraries above. Some sample usage:

from pathlib import path

# Declaring a path object is just passing the path string
p = Path("/home/roytang/stuff")

# Other paths can be derived using the / operator
subdir = p / "favorites"

# p.glob replaces stuff like os.walk and glob.glob
for mdfile in p.glob("**/*.md"):
    print(str(mdfile))

    # path objects have properties for individual path elements
    filename = mdfile.name
    stem = mdfile.stem # filename without extension

    # path.exists() replaces os.exists()
    newdir = subdir / stem 
    if not newdir.exists():
        # path.mkdir replaces os.makedirs
        newdir.mkdir(parents=True)

    newfile = newdir / newfile # path objects can be either files or dirs
    # for file copying, you still need shutil!
    import shutil
    shutil.copy(str(mdfile), str(newfile))

For me it’s much cleaner to use than the old methods and require less imports (most of the time). I eventually hope to migrate all my older scripts to use Pathlib (the same way I migrated them away from Python 2 a while back), and moving forward I plan to use it primarily for filesystem operations.

TriviaStorm: Text and Answer parsing

hello@roytang.net (Roy Tang) — Sun, 03 Feb 2019 05:56:56 +0000

A while back I started a Twitter trivia bot as a weekend project. That bot is still up and running on Twitter, you can check it out there!

But today, I thought I’d write about the answer-checking mechanism used by the bot. It was a bit interesting to me because it was the first nontrivial use I had for Django’s unit testing framework. I’m not too keen on unit testing web functionality (something I still have to learn), but this seemed an appropriate first use of a unit test framework for several reasons:

the bot had to be able to handle a wide variety of answers
there were a lot of test cases to check and a single checking function handling everything - I couldn’t risk breaking previous working tests
inputs were discrete and outputs were easily checkable
I needed to be able to add new test scenarios all the time as more problematic answers were provided

The project currently isn’t open source, but I did make a gist of the tests.py I used here.

The check function basically accepts three parameters:

a checking mode (currently only supports EXACT and ALL_ANYORDER)
the answer phrase provided by the player
a set of valid answers accepted for the question

There’s a number of test cases already handled:

checking should be case-insensitive
articles should be ignored if they’re at the start of the answer phrase
numbers should be acceptable for the spelled-out versions i.e. “7” should be accepted for “seven”, and vice versa
some minor soundex (phonetic matching) support (via Python Jellyfish)
handling of questions that support multiple answers. This is what ALL_ANYORDER is for - it means that all the given answers must be provided, but they can be in any order. i.e. if the valid answer set is "Huey", "Dewey" and "Louie", then "Louie Huey Dewey" should be accepted as an answer
nonalphanumeric characters should be treated as whitespace, except in some special cases
special case: abbreviations like "don't" or "can't" should be treated as if they were a single term like "dont" or "cant" instead of "don t" or "can t"

The answer checking definitely still isn’t perfect, but I’m pretty happy with where it’s at right now. There is also definitely an element of subjectivity as to which answers should be accepted. One time a player complained that his answer "Batman vs Superman Dawn of Justice" should count for "Batman v Superman: Dawn of Justice", but for this particular question I had chosen not to allow “vs” for “v” because that was the actual movie title, which might be unreasonable now that I think about it!

I do know that I need to implement a better “synonym” handling, i.e. mapping of "v"<->"vs" and other terms like "mr"<->"mister" or "natl"<->"national". The problem with handling things like that is that is the possible combinations of phrases expands when multiple such terms are found in the same answer, so it can’t scale too well. I suppose I need to normalize the answer sets at the time the question is defined. What do you know, I figured out how to do something just by writing a blog post!

I do have a bunch of other enhancements planned for the trivia bot, including support for slack and discord, and a longer time frame roadmap, but I’m not sure when I can commit more time to it. Still, it’s turned out to be a pretty fun endeavor, I’m hoping it leads to something cool!

Django Blog Application

hello@roytang.net (Roy Tang) — Sun, 28 Oct 2018 05:02:37 +0000

Ten years ago this month, I started studying Django by trying to build my own blog application. I found the code lying around while I was going through some backups lately. It’s way out of date, it uses an early version of django. I thought of bringing it up to speed, but that didn’t seem practical. Instead, for archival purposes, I cleaned it up a bit and uploaded the code to a github repo. (Helpful github immediately warned me that having a very old version of Django was a security risk lol). There’s a lot more information in the README.md of that repo. I actually used this as my main blog engine for a while before I decided the maintenance effort wasn’t worth it and switched to WordPress.

Tangent: I’m not actually 100% happy with WordPress, and when I found this old code I was tempted to maybe trying building Yet Another Blog Application (TM), except maybe using the opportunity to learn some other framework. My 2018 personal to-do list is already way too long though.

Running Python 2.x and 3.x on Windows

hello@roytang.net (Roy Tang) — Thu, 02 Mar 2017 01:30:38 +0000

I’ve been hesitant to try Python 3.x because it’s not backward compatible with Python 2.x which I’ve been using for scripting since forever. But recently I found out that since Python 3.3, they’ve included a launcher in the Windows version that supports having both versions installed.

You can use the launcher to specify the Python version to use at the command line (it defaults to whichever version was installed first):

(Click to view full-size)

Even better, the launcher will recognize a header in your .py files that can specify which version of python to use:

#!/usr/bin/env python3

If the launcher sees this header, it will automatically launch the appropriate python version. Handy!

Weekend Project: Twitter Trivia Bot

hello@roytang.net (Roy Tang) — Thu, 23 Feb 2017 01:30:00 +0000

I had been meaning to try writing a Twitter bot for a while now. I figured a trivia bot would be pretty easy to implement, so I spent some time a couple of weekends to rig one together.

It’s (mostly) working now, the bot is active as triviastorm on Twitter, with a supporting webapp deployed on https://triviastorm.net/. The bot tweets out a trivia question once every hour. It will then award points to the first five people who gave the correct answer. The bot will only recognize answers given as a direct reply to the tweet with the question, and only those submitted within the one hour period.

Some technical details:

My scripting language of choice for the past few years has been Python 2.7. I’m using Tweepy to interact with the Twitter API, PyMySQL to connect to the database, and Flask to run the webapp. I haven’t used Flask in some time, but it’s still very straightforward. I actually had a harder time configuring the webapp with mod_wsgi on my host.

The main problem with a trivia system is that you need a large and high-quality set of questions. Right now the bot is using a small trivia set –around a thousand questions I got from a variety of sources. If I want to leave this bot running for a while, I’m going to need a much larger trivia set. However, reviewing and collating the questions is a nontrivial task. Hopefully I can add new questions every so often.

Feel free to follow the bot and help test it out. I’d be grateful!

The Simplest Code That Can Do The Job

hello@roytang.net (Roy Tang) — Wed, 11 Jan 2017 01:30:06 +0000

So the other day I was reworking a Python script that I had been using for years on my home PC to manage and categorize some downloaded files for me. This time I wanted to add some smarter behavior to make it more able to figure out when to group files into folders without constantly needing manual intervention from me. To do this, I needed to persist some data between runs – so that the script remembers how it categorized previous files and is able to group similar files together.

Now since my software development career has largely been as an enterprise-y kind of developer, my first thought was to just use a database to store the data. I already had a MySql installation on my machine so that was fine, I just needed Python to interface with it. After looking up how to do it, I balked at having to install a new Python library just to connect to MySql and reconsidered.

As programmers, we have a tendency sometimes to over-engineer solutions because that’s what we’re used to doing. Did I really need a database for this? The data won’t be very big, and I won’t need to do any sort of maintenance on it, so maybe a simpler solution was in order.

I ended up just using pickle, which was already built-in to Python:

def load_db():
	all_series = {}
	with open(DATABASE_FILE, 'rb') as handle:
		all_series = pickle.load(handle)
	return all_series

def save_db(all_series):
	with open(DATABASE_FILE, 'wb') as handle:
		pickle.dump(all_series, handle, protocol=pickle.HIGHEST_PROTOCOL)

(Above code probably gives you an idea what kind of files I’m sorting… )

As an added benefit, I didn’t need to design any database schemas or tables or whatnot, pickle just lets me serialize the map as-is and reload it later from disk without any hassle.

I guess my lesson here was: don’t over-complicate things when something simple will work fine. Write the simplest code that can do the job.

Reply to 'I'll be your python slave. Looking a py project to work at to become better.' on /r/Python at https://reddit.com/r/Python/comments/jw5lx/ill_be_your_python_slave_looking_a_py_project_to/:

hello@roytang.net (Roy Tang) — Sat, 27 Aug 2011 00:00:00 +0000

Reply to 'I'll be your python slave. Looking a py project to work at to become better.' on /r/Python at https://reddit.com/r/Python/comments/jw5lx/ill_be_your_python_slave_looking_a_py_project_to/:

Earlier today i was thinking of writing a python script to detect duplicate mp3s, you might want to try that. There are existing programs to do it, but they all seem terrible!

New note

hello@roytang.net (Roy Tang) — Sun, 17 Oct 2010 03:01:52 +0000

In Python, say I have a string that contains the name of a class function that I know a particular object will have, how can I invoke it?

That is:

obj = MyClass() # this class has a method doStuff()
func = "doStuff"
# how to call obj.doStuff() using the func variable?

Reply to 'Python execution speed: laptop vs desktop' on stackoverflow at https://stackoverflow.com/questions/3939912/python-execution-speed-laptop-vs-desktop:

hello@roytang.net (Roy Tang) — Fri, 15 Oct 2010 06:27:52 +0000

Reply to 'Python execution speed: laptop vs desktop' on stackoverflow at https://stackoverflow.com/questions/3939912/python-execution-speed-laptop-vs-desktop:

Split your processing into multiple threads. Your particular i7 should be able to support up to 8 threads in parallel.

Ask HN: How will it impact my webapp if it is written in Python instead of PHP.

hello@roytang.net (Roy Tang) — Tue, 06 Jul 2010 12:19:00 +0000

Shared via delicious: Ask HN: How will it impact my webapp if it is written in Python instead of PHP.

One of the responses has a nice discussion on the state of python web frameworks.

New note

hello@roytang.net (Roy Tang) — Wed, 16 Jun 2010 07:12:12 +0000

Is there any Python library that allows me to parse an HTML document similar to what jQuery does?

i.e. I’d like to be able to use CSS selector syntax to grab an arbitrary set of nodes from the document, read their content/attributes, etc.

The only Python HTML parsing lib I’ve used before was BeautifulSoup, and even though it’s fine I keep thinking it would be faster to do my parsing if I had jQuery syntax available. :D

Anyone care to share neat python tricks? : programming

hello@roytang.net (Roy Tang) — Wed, 13 Jan 2010 00:46:28 +0000

Shared via delicious: Anyone care to share neat python tricks? : programming

Reply to 'ValueError in Django' on stackoverflow at https://stackoverflow.com/questions/2032360/valueerror-in-django:

hello@roytang.net (Roy Tang) — Sat, 09 Jan 2010 04:47:32 +0000

Reply to 'ValueError in Django' on stackoverflow at https://stackoverflow.com/questions/2032360/valueerror-in-django:

urlpatterns = patterns('',     
    (r'^salaries/employee/$', list_detail.object_list, 'employee_info'),
)

The third item in the tuple needs to be a dictionary, not a string. Try removing the single quotes around employee_info:

urlpatterns = patterns('',     
    (r'^salaries/employee/$', list_detail.object_list, employee_info),
)

Reply to 'How to create a .py file for Google App Engine?' on stackoverflow at https://stackoverflow.com/questions/1866147/how-to-create-a-py-file-for-google-app-engine:

hello@roytang.net (Roy Tang) — Tue, 08 Dec 2009 10:52:08 +0000

Reply to 'How to create a .py file for Google App Engine?' on stackoverflow at https://stackoverflow.com/questions/1866147/how-to-create-a-py-file-for-google-app-engine:

You will need a better editor than Notepad. With Notepad, use Save As…, and type “helloworld.py” in the dialog, including quotes so that the file extension is .py instead of .txt

Reply to 'Is there a tuple data structure in Python' on stackoverflow at https://stackoverflow.com/questions/1831218/is-there-a-tuple-data-structure-in-python:

hello@roytang.net (Roy Tang) — Wed, 02 Dec 2009 07:37:51 +0000

Reply to 'Is there a tuple data structure in Python' on stackoverflow at https://stackoverflow.com/questions/1831218/is-there-a-tuple-data-structure-in-python:

You can have an array of 3-item tuples.

arr = [ (1,2,3), (4,5,6), (7,8,9)]
for (k, v, x) in arr:
  # do stuff

Andy McKay's blog

hello@roytang.net (Roy Tang) — Sun, 06 Sep 2009 04:53:07 +0000

Shared via delicious: Andy McKay's blog

Django snippets: Facebook shell

hello@roytang.net (Roy Tang) — Sun, 06 Sep 2009 04:52:51 +0000

Shared via delicious: Django snippets: Facebook shell

Getting Started - wxPyWiki

hello@roytang.net (Roy Tang) — Mon, 13 Apr 2009 15:04:33 +0000

Shared via delicious: Getting Started - wxPyWiki

Stylesheet Switcher using Django

hello@roytang.net (Roy Tang) — Sat, 14 Feb 2009 14:17:32 +0000

You may have noticed the new color scheme and new “Theme Switcher” widget in the sidebar. I had done some CSS work during the past month in the office and it made me want to tweak the stylesheets on this site a bit. I figured I might as well make it easy to switch stylesheets, so I wrote a small Theme Switcher django app. (Well, it’s more of a stylesheet switcher I guess)

The model is simple:

class Theme(models.Model):
    slug = models.SlugField()
    title = models.CharField(max_length=250)
    css_path = models.CharField(max_length=500)

    def __str__(self):
        return self.title

I only needed one view – to switch the current theme.

Added the ff to urls.py:

(r'^theme/([a-zA-Z0-9\-]+)/', 'themer.views.switch'),

The view code simply stores the selected theme as a cookie:

def switch(request, theme_slug):
  """ Sets a cookie to specify the new theme, then redirects to the referer or root url """
  theme = get_object_or_404(Theme, slug=theme_slug)
  referer = "/"
  if "HTTP_REFERER" in request.META:
    referer = request.META["HTTP_REFERER"]
  resp = HttpResponseRedirect(referer)
  resp.set_cookie("theme", theme_slug, 100000)
  return resp

To load the stylesheet, I set up a context processor to pass both the current theme and the list of themes to the context. I wanted to do this using template tags, but I couldn’t figure out how to extract a cookie from within a template tag.

from themer.models import Theme
def context_processor(request):
    theme_list = Theme.objects.all()
    if "theme" in request.COOKIES:
        slug = request.COOKIES["theme"]
        try:
            theme = Theme.objects.get(slug=slug)
        except Theme.DoesNotExist:
            theme = None
    else:
        if theme_list.count() > 0:
            theme = theme_list[0]
        else:
            theme = None
    return {"current_theme" : theme, "theme_list" : theme_list}

Then in my base template file, I add the following to the head section:

<link rel="stylesheet" href="{{ current_theme.css_path }}" type="text/css" />

And add the list of themes into the sidebar.

<h2>Theme Switcher</h2>
<ul>
            {% for theme in theme_list %}
            <li>
            {% ifequal theme.slug current_theme.slug %}
            {{ theme.title }} <span class="note">(current)</span>
            {% else %}
            <a href="{% url themer.views.switch theme.slug %}" title="Switch to {{ theme.title }} theme">{{ theme.title }}</a>
            {% endifequal %}
            </li>
            {% endfor %}
            </ul>

And we’re done!

Of course, I’m still not particularly strong in web design, so the two current “themes” really just switch around the color scheme. The light blue theme is named Azorius, while the old black and green theme is named Golgari, both named after the respective color guilds in Magic: The Gathering’s Ravnica block.

Hopefully this gives me a chance to whip up more interesting themes and polish the old CSS skills some more. Enjoy!

Using Django Pingback

hello@roytang.net (Roy Tang) — Sun, 04 Jan 2009 07:22:20 +0000

I actually had some trouble using django-pingback on my custom blog engine; the django-pingback documentation is mostly fine, but there were some caveats that I had to discover myself through a bit of debugging:

The URL specified for the XML-RPC endpoint in the HTML head needs to be a full absolute url including domain, i.e. http://roytang.net/xmlrpc/, which gave me trouble when I was trying to test using localhost pinging to an online server. I eventually just decided to set it up, deploy on webfaction and test it online before I redirected the domain name.
The documentation mentioned that the pingback handler needs to have the same arguments as the detail view for the post, so I wrote it as follows:
```
# exactly same arguments as 'details' view.
def pingback_blog_handler(catslug, year, month, slug, **kwargs):
    cat = Category.objects.get(slug=catslug)
    return Post.pub_objects.get(date__year=year, date__month=month, slug=slug, category=cat, published=True)
```
But it wasn’t exactly working for me, for some reason the function wasn’t being called. After some debugging I found that I needed to use named parameters for catslug, year, month, slug in the URLConf for the post permalink in order for this to work. I’m not sure if this was because I was doing it wrong or it’s some sort of undocumented requirement.

The pingback client was throwing a KeyError whenever a post I made would have anchor tags without an href attribute. Granted that there isn’t much point in having an anchor tag without the href, but the code shouldn’t just choke on it. I modified the following line in client.py:
```
  links = [a['href'] for a in soup.findAll('a') if is_external(a['href'], url)]
```
Adding the check for the href attribute:
```
  links = [a['href'] for a in soup.findAll('a') if a.has_key('href') and is_external(a['href'], url)]
```
Those are all the changes I made to get pingbacks working. I hope they’re still working now, as I haven’t received any pingbacks since the django version of this blog went live >.

Colophon 2008

hello@roytang.net (Roy Tang) — Sat, 27 Dec 2008 02:08:22 +0000

Frontend: All page templates are valid (X)HTML. However, I choose not to claim valid XHTML (and no doc type declaration) since I can’t guarantee that blog posts I write are compliant! The site uses standard CSS and uses the Blueprint CSS Framework for the grid layout of the page. The site design is entirely original (if not simple and bland – I’m not very good with website design yet!). The site has minor usage of JQuery Javascript library in some parts.

Backend: The server side uses Python and Django behind mod_python on Apache, with a MySQL database. The following Django apps and Python libraries are used:

custom made blog application by myself
built-in (contrib) apps: admin, comments, sitemaps, flatpages
django-tagging
django-xmlrpc
comment_utils for advanced comment moderation features
template_utils
django-pingback
Beautiful Soup for XML/HTML parsing
Pygments for syntax highlighting. See the Pygments Demo!
Feedparser
Markdown2 so that I can post using Markdown.

As is typical for hobbyist software development, I started working on this site with only a minimal set of features considered, but as I kept working I kept wanting to add more stuff. Some notable custom-made features:

XMLRPC server, supporting Metaweblog API and Pingbacks. I actually only added the metaweblog support so I could post from Flock
Autocard support (same as my previous Autocard plugin for WP) and syntax highlighting of code.
Automatic extraction of twitter and delicious entries into the sidebar
Gravatar support for comments

All entries from the previous “roytang / weblog”, “Roy on Magic” and “Roy on Django” have been imported using a custom data conversion script, but nothing is perfect so some of the older posts may be badly formatted. (Comment on those posts!)

Hosting: Provided by Webfaction.

Fixing up the comments

hello@roytang.net (Roy Tang) — Mon, 01 Dec 2008 04:58:00 +0000

I played around with the built-in comments app today, trying to clean it up.

Some findings:

Some of the moderation views, such as flagging a post or deleting a post, accept a next parameter that determines where the view will redirect to after the operation. However, the way the url’s are set up, there’s no easy way to pass this parameter normally, even through query strings. This lovely bug is documented in http://code.djangoproject.com/ticket/8968. I wasn’t confident enough to try patching it, so as a temporary workaround, I overrode the flag and delete templates and placed the following inside the form:

<input type="hidden" name="next" value="{{ comment.get\_content\_object_url }}" />

Edit: Wow that was stupid. Blogger didn’t want to render the HTML code above!

I also replaced the URL in the “cancel” links for both pages. For some reason, {{ comment.permalink }} wasn’t giving me anything useful.

Took me a bit of work to figure out how the moderation part works. Apparently I have to hook on some of the signals to add my moderation logic. I’m not sure yet how to handle this, or whether I should try customizing the comment form with something like a captcha to prevent comment spam.
I have a pending problem now that submitting comments from the preview or error page does not redirect to the correct place. (i.e. the post page). I tried adding the next parameter to the form, but apparently comment.get_content_object_url returns an empty string here because the comment has not yet been saved to the database. I think the content object should be provided in the context for the preview page. This would be handled by http://code.djangoproject.com/ticket/9268, except apparently no one’s paying attention to it yet.
Some random features I added: Gravatar support (the Gravatar for the email address used to post the comment will be shown), and Markdown support in the comment body. Easy stuff.
Cleaned up an refactored the templates that render the comments, separating them from the parts that render the blog posts. This makes it easier if I want to have comments for other non-Blog post objects later.

Non-comment related:

Added simple search!
The script to import from WordPress has some problems. Since I’m storing the post body in markdown format, I use html2text to convert the HTML body from old posts into markdown, but nice stuff like image alignments and tables are lost. I think I have to make do with editing straight HTML for old posts imported from WordPress, and just clean them up later as needed.

My code is rather messy now (especially from the HTML side). I should clean it up sometime soon.

Django Update -- WordPress Import and More

hello@roytang.net (Roy Tang) — Sun, 23 Nov 2008 00:04:00 +0000

I’ve been busy at work so fell a bit behind with Django. Last night I worked on a WordPress importer, so that I could migrate posts from my current blog(s) into the Django-powered blog that I’m coding. I’m using BeautifulSoup to parse the WordPress export file and insert them as Django objects.

Since I was running the script repeatedly, I had to figure out how to easily run it from the command line, without having to run it from inside manage.py shell. After some searching, I found that what I needed to do was set an environment variable DJANGO_SETTINGS_FILE to point to my settings file. After that, the importer script could be run repeatedly.

Also made a few tweaks to my dev copy:

Modified URL mapping to follow the permalinks of the old blog (so that people linking to me don’t suddenly have broken links)
Added a tag cloud (Still using django-tagging)
Added markdown support to the admin maintenance screen. I used to think I would need to code my own form for posting, but I think I can live with just using the admin-provided one (initially at least)

I plan to migrate my main blog over once I take my Christmas break (hopefully 2nd week of December)

Stuff to do:

Use clean, non-hardcoded urls (url template tag and reverse())
Customize the admin screen for posting
Clean up the layout and templates

django-tagging

hello@roytang.net (Roy Tang) — Sun, 26 Oct 2008 04:22:00 +0000

I wanted to add some basic tagging to my blog app so I tried out django-tagging. Unfortunately, the featured downloads on the Google Code site are quite out-of-date and would not work with Django 1.0, so I did a subversion checkout instead. If you’re getting an error like “ImportError: cannot import name parse_lookup”, then you need to get the source code from SVN.

Adding the tagging to the blog was pretty easy:

Add the tagging app to settings.py
Add a tagging.fields.TagField to the Post model
Add a “tags” text field to the post form used.
Modify templates to display the tags.
I used something like “/tag/” url mapping to get all posts associated with a tag. Then you just need to write a wrapper around the object_list generic view:

from tagging.models import Tag, TaggedItem

from django.views.generic.list_detail import object_list

def posts_by_tag(request, tag):

o_tag = Tag.objects.get(name=tag)

queryset = TaggedItem.objects.get_by_model(Post, o_tag)

return object_list(request, queryset)

This view will use the same template used to list out posts normally.

I left it for a while

hello@roytang.net (Roy Tang) — Fri, 24 Oct 2008 06:54:00 +0000

… and now it works!

http://www.randomencounters.bells-n-whistles.net/blog/

I think the apache service needed to restart mod_python or something… (I don’t have SSH access so… )

Deployment Problems

hello@roytang.net (Roy Tang) — Tue, 21 Oct 2008 06:52:00 +0000

So I got a basic blog app up and running.

Posting, paged archives, etc.

Comments implemented using the django.contrib.comments. No problems here, the only caveat being most of the current documentation found by Google searches refer to the pre-1.0 version. Need to peruse the official docs for 1.0 stuff.

RSS feeds implemented using django.contrib.syndication, this one seems fine.

I tested it and it’s running fine on localhost. I also have a free django hosting account at http://bells-n-whistles.net, so I try to upload it there. However, when I access the website there (http://www.randomencounters.bells-n-whistles.net/blog/), I get the following error:

'comments' is not a valid tag library: Could not load template library from django.templatetags.comments, No module named comments

I think the settings.py is not being reloaded - I tried to set Debug = False and I’m still getting the stacktraces.

Bah, I’ll figure it out tomorrow.

Playing with Generic Views and URLs

hello@roytang.net (Roy Tang) — Mon, 20 Oct 2008 13:44:00 +0000

“when redirecting, how can I make the redirect URL decoupled from the urls.py of the parent app?”

-> It turns out that HttpResponseRedirect supports relative paths, so this was fine.

return HttpResponseRedirect("../" + str(post.id) + "/")

I got the basic posting structure up.

/post/new/ -> To make new posts

/post// -> To view a single post

/post/all/ -> To view all posts

I should probably start thinking of a better url scheme. Ideally, I’d want the @login_required views to be indicated as such in the urls. Something like “/admin/post/” for new posts “/admin/manage/” for a screen to manage posts to differentiate it from a screen just to list them out.

Next I think I’ll try to CSS-ify the blog; I’ll probably just reuse stuff from one of the blogger templates as I’m still not very good with the HTML/CSS.

I’ll also look into using some of the date-based generic views to get an archive view. I’ll need more data though. Will consider importing from the WordPress blog.

Starting out

hello@roytang.net (Roy Tang) — Sun, 19 Oct 2008 07:33:00 +0000

The quintessential app to learn from is of course a blog.

Started using a simple Post model. Added the new post form and view. Can now successfully insert posts into the DB.

– create the detail page that will show the post after saving

Figure out:

– when redirecting, how can I make the redirect URL decoupled from the urls.py of the parent app?

i.e. if the parent app has the following mapping:

‘^blog/’ -> pass to blog.urls.urlpatterns

the blog app has mappings for

‘^post/new/’ -> new post

‘^post/([A-Za-z-])/’ -> post detail

inside the view, I want to redirect to “post/”, but with respect to the app, it should be “blog/post/”

I should subscribe to one of the Django mailing lists to ask stuff like this.

Write Facebook apps using Google AppEngine -- Patrick Crosby

hello@roytang.net (Roy Tang) — Sun, 06 Jul 2008 15:21:24 +0000

Shared via delicious: Write Facebook apps using Google AppEngine -- Patrick Crosby