The new nearest stops card

Okay so it’s been a long time since the last update, around a month. Mainly it’s because I haven’t actually made much progress – vacation over Easter and then moving for another two weeks. I have made progress but only really starting this week.

One thing I’ve worked on is adding a view in the app where you can see all trips between one stop and another. I have that sort of working now, as shown in the video, but there’s a lot of work left polishing it and testing that the data shown is correct.

Something else I’ve been wanting to look into is getting a better understanding, by which I mean giving the app a better understanding, of the data it’s working with. Right now it only knows that there are stops and trips between them but it doesn’t understand that, for instance, there are things called cities and towns that those trips take you between. When I’m going to say Middelfart I don’t think of it as going a train station called “Middelfart Station (Middelfart Kommune)” – I think of it as going to a city called Middelfart. And when I’m going to Trøjborg I don’t think of it as going to the bus stop “Tordenskjoldsgade/Herluf Trolles Gade (Aarhus)” I think of it as going to a neighborhood called Trøjborg. The data I have though only knows about the bus stops and train stations, so I’ve spent some time building some more detailed information about where the stops are. That’s included looking up the nearest address to each stop, for instance, from google which sometimes tells you the area a stop is in. I’ve also extracted lists of villages, towns, cities, neighborhoods, etc from open street maps. Along with the information I already have, at this point I think I have all the data I need for the app to understand the higher-level geographic concepts. I’m not going to use it quite yet but now I have it.

The last thing I’ve been working on is a hairy issue that I’m happy to finally have a solution to – but creating that solution has been hard going. It’s a subtle but important issue. It’s about what to call stops, stations, routes, and the other “entities” that you deal with in transit schedules. The app stores all the schedules in a file called a “bundle”. Over time the schedules change and when that happens the app receives a new bundle file and throws away the old, now obsolete, one.

All stops and routes have an id in the data I get from rejseplanen. For instance, the stop id for Aarhus Rutebilstation is 000751000100G and the route id for Endelave - Snaptun is 21063_4. I throw those ids away though when I build the bundles that the app uses because they take up a lot of space. Instead I replace them with a simple number, so within the bundle the stop might be called 12620 and the route 247. Besides saving space the numeric ids solve a bunch of other problems that makes the app use less memory and faster. So all’s well right?

No. Here’s the problem: the number for a given entity doesn’t stay the same across bundles – they get generated from scratch every time the schedule changes. So Aarhus Rutebilstation will be 12620 now but after the next update it might be 11394. That’s fine in most cases because you only use one bundle at a time – but “most cases” isn’t always. If for instance you want to save a shortcut in the app that takes you directly to departures from a particular stop I can’t store the numeric id because then the shortcut will break when you update and the ids change. I also can’t store the name of the stop because there’s no guarantee they won’t also change – names of things change. I can’t store the id from the original data either, the 000751000100G, because I’ve thrown that away. So what to do?

The solution I’ve come up with keeps the numeric ids exactly as they are but introduces a new kind of id that is guaranteed to stay the same across updates. So now Aarhus Rutebilstation will have three ids: the one it has in the original data, 000751000100G, which is fixed but takes up too much space. It also has the numeric one it has within a single bundle which is fast to use and saves space but which changes between bundles, that’s the one that is used for almost everything. But now there’s also a new one which is less efficient to use but takes up very little space (like less than 0.1% of the other id) and which never changes.

The reason I’ve had to be really careful with this is that as soon as I start using this new kind of id I’m stuck with it – the whole point is that it doesn’t change so if I mess it up it’ll be really super hard to fix. So that’s taken a while. But I think I’m there now. Almost. Just one more test. Then I’ll be done. Almost.

With that problem solved I can get back to the main thing I want to implement: storing shortcuts to particular schedules on the main page. Easy access to the schedules you use most is essentially the whole point of the app so that’s a core feature and I’m really eager to make some progress on that – my goal is to have some version of that early next week.

Since last I’ve added an initial implementation of showing trips between a given start and endpoint. Also searching through possible endpoints works, sort of. (The app is really super slow in the recording because I was running a huge job on the same machine at the same time. D’oh!)

Since my last update I’ve spent time on three different things.

The first thing is that all my source code used to be publicly visible – anyone could find and read it all. It was just easier to to start out that way. But at this point I thought it best to make it all private, mainly because I have things that I’m not allowed to distribute – like configuration files derived from timetable data that doesn’t allow derived works to be distributed – that I’d like to have as part of the source code. It took a fair amount of work because I have to reconfigure a bunch of things, but that’s done now.

Secondly I’ve been working on text search, basically searching for stops and routes by their name. There’s a lot of stops and I want searching through them to be fast – I want to show the search results immediately as you type into the search field. Figuring out how to support that that doesn’t require huge tables has taken some effort but I’ve come up with an approach that might work and have done most of the work of implementing it. What’s missing now is adding the search UI in the app. That’s why I say it might work, not that it does work, because I haven’t actually tried using it in practice so I don’t know if it’s fast enough yet. But that leads me to the third thing I’ve been working on.

I’ve now mostly implemented the UI I designed a couple of weeks ago and I have some new features where all that’s missing is implementing the UI. So I’ve been designing some new UI stuff, notably searching and showing both ends of a trip, both from- and to-, in the view. I also need a design for showing an individual trip with all the stops but I haven’t gotten to that yet.

The rest of this week I’ll work on implementing search and related things in the UI.

Since last I’ve made a lot of progress on the UI. Here’s what the app looks like now.