- Home
- blog
- flickr
- twitter
- github
- youtube
- last.fm
- pe@ppy.sh
-
Posts
- developing an allergy to apples
Recently I have been heavily involved in Apple platform development — specifically of the mobile variety. This has involved launching and maintaining a prominent e-book reader (with a user base of 500k+) alongside my personal projects’ apps (puush and osu!stream). While I generally try to see the best of any situation, developing for iOS has fueled stress and anger that I am not used to, and even made me consider at times that changing profession wouldn’t be such a bad thing.
And don’t get me wrong, I love coding. I live for code. So something must be drastically out of balance here, right? While some of the issues I have are specific to my own workflow and methodology, I think most of you will be able to agree on some level that things could definitely be greener in the Apple ecosystem. I’m going to touch on a few specific areas of iOS app development over the last couple of years which have hampered my productivity. Hopefully it will serve as a light warning to those looking at entering this arena.
Apple always has the last say
Since releasing our e-book reader, we have had to remove over half the functionality in order to keep it live on the store. After Apple decided to enter the e-book market, it was initially unclear as to how we would be affected; I knew there would be a drop in sales as Apple leverages their App Store and OS strongholds to promote their new app and store, but they did not stop at just that.
In a manner that I would personally say is many times worse than the age-old Microsoft anti-trust case, Apple gradually changed their policies to lock competitor e-book providers out of their system. It is obvious that they were not happy having customers using their devices to purchase books via other providers (even though all fulfillment was done via an external web link), and gave us a deadline to make the transition to In-App Purchases. This would mean giving a 30% revenue margin to Apple.
While reluctant to take this approach, we did end up attempting to embrace Apple’s restrictions. Sadly, the publishing industry is a very complex one, and payments and pricing could simply not be moulded into Apple’s tier-based pricing system. This would not allow for regional pricing, state-based tax, 100k+ product listings and rapid price changes, to name a few problematic points. Of course, Apple would have understood this from the beginning — having to abide to the same contractual terms with publishers themselves when distributing via iBooks — and pushed the deadline forward further while they planned the road forward for their competitors.
Then, along with Amazon and other key e-book providers, we were given an ultimatum: remove the store completely from the app. “Okay”, we said, and replaced the in-app store with a link to our website. This was not enough to please apple; within a day our app submission was rejected, and what followed were a few months of back-and-forward build rejections. The end result wasthe removal of every UIWebView from our application, along with every external link. Yes, we could no longer provide even a web link taking customers to our company’s own web site (not even a “forgotten password” link!).
The end result is a thin-client reader which can do not a lot more than read and sync books purchased outside the iOS environment. Potential customers cannot make a purchase unless they manually make the connection via our app and our website.
Too late to apologise
When I initially made osu!stream available, I ran into some licensing complications with the music I was using, resulting in the need to quickly pull the app (of my own accord) and replace the songs. A month later, I submitted an update to the app, which amongst many other improvements, mapped the previous song packs to newer ones. This was a way of ensuring users which bought the initial song packs didn’t feel cheated after they were made inaccessible.
To reach the maximum amount of users possible with instructions on how to retrieve the new content (and providing a brief explanation of the events that took place), I wrote up release notes which had a paragraph dedicated to the licensing issues. In this paragraph, I apologised for what happened. This update got rejected on basis that release notes were “not related to the changes in the app”.
When Apple reject an update, they generally do not give any specific instructions, instead only quoting the relevant sections from the Terms of Service passage and leaving the rest to you to figure out. I had a hunch that the word “sorry” triggered alarm bells, so changed only that sentence; accordingly, it was approved one day later (n.b. they did actually check each localisation, so it took two shots!).
While this might not seem like a huge deal, it is just another example of the kind of randomly exerted control they have over the approval process.
As agile as a snail
One of the more apparent downsides of iOS development is the compulsory approval process associated with deploying an app to the App Store. Depending on seasonal demand, this approval process can take anywhere between 6 hours and 14 days (based on my experiences to date). The lower end of the scale is generally only seen when it is in Apple’s interest to get it live quickly.
I usually work with a very rapid release cycle, usually aiming for daily releases — pushing features and fixes the moment they are mature enough. At times this means relying on the end-user to do the “testing”, but I find this to be the most productive release method. Keeping things fresh generally keeps people happy and coming back for more. Maybe I am designing for myself here, but if I was a user, I would enjoy seeing my feedback rapidly incorporated in a product.
Being limited by external forces to an average of weekly releases — with no rapid user feedback in the interim — makes development very staggered and unproductive for me. This is likely amplified due to the way I work, so I would be interested to hear other people’s experiences with the imposed time-between-release limitation.
You might say “but you can deploy to testers as often as you want via a rapid deployment process (testflight is awesome)!”, to which I would answer that I am definitely aware of this and actively deploying regular builds using these methods. The main problem here is that each developer account is limited to 100 registered devices for device provisioning, and while this may sound like a large number, I found that a month or two after accepting testers over 80% of them were inactive. You are only given one opportunity each year to remove a number of devices (at the anniversary of your account registration), which is both cumbersome and sub-optimal.
There are also many documented examples of extreme cases where apps get stuck in a pending state with absolutely no response from Apple. 2Do is one current example, who seem to be stepping in dangerous territory due to their iCloud’s CalDAV sync. I have had some personal experience in trying to get a response from Apple’s chain-of-command via phone and email and can attest that it is not so easy. Lack of control over your own product’s release cycle — and a communication blackout from those in control — is indeed quite a scary concept.
Think Different, or Don’t Think at all?
The final straw (which actually triggered the writing of this article) was having my app rejected today for storing downloadable content in the application’s “Documents” folder. Since the introduction of iCloud, the guidelines have been changed to only allow “user-created” documents to be stored in this specific folder. This wouldn’t be such a bad thing if it wasn’t the only non-volatile storage area available to developers.
In short, this means that if myself and other developers in a similar predicament are to abide with the new terms, iOS can choose to delete all downloaded content in any app it sees fit to free up memory. Users are not given the option to disable this behaviour — let alone warned about it when it happens. This is a support nightmare for app developers, who users are going to obliviously blame for the loss of their files.
The new “cleaning” behaviour (more details in this article) is going to surprise many users over the coming months as their files disappear at iOS’ whim. In osu!’s case, this means all purchased songs would disappear, requiring special support at an application level to track when files have disappeared, alert users and trigger a fresh (and still quite awkward) in-app purchase process to re-retrieve the content. With a bit of luck, the user will have an internet connection available at this point, but what happens if they are in the middle of a long flight, or in an area without coverage?
I have been following this behavioural issue since it was discovered earlier this month, but didn’t think I would be affected — after all I am only storing around 10mb (at most) to the Documents folder. It would seem that for whatever reason, Apple is not only actively enforcing this new clause, but cracking down on it hard.
Waiting a week for approval to only be rejected is quite a bad feeling (especially when you have other coinciding release details planned), but when the rejection is due to this kind of issue it really makes you wonder if Apple know what they are doing. And makes me ask myself whether I really want to continue developing for a platform this unstable.
—
Hopefully this didn’t come out as too much of a rant — I do always try to look at things from a neutral perspective. I won’t be stopping iOS development, but may take a few days off to regain some motivation :). I am not changing my app’s behaviour to meet the new clause requirements and have instead filed an appeal with Apple. I hope there is enough velocity from my appeal, along with other affected developers/users’ voices to change the way things work.
- Fixing the Parallels Dock Icon
So I have been using parallels for a while now, and it is turning out to be a very good experience. I would like to write a longer post on the ups and downs of the application itself, but this post is a solution to a long-standing gripe I’ve had with the product line — the dock icon.
“Paused? No, that’s our logo. You can’t remove it, either.”
I’m sure you can agree that not only are the Parallels bars ugly, but they also communicate a state which is actually incorrect. The resemblance of a pause symbol is undeniable. Having this icon in my dock is an eyesore and a distraction due to the fire-truck red colour.
Fixing it is unfortunately not so straight-forward, and while many people have attempted to cry for help, those behind parallels have refused to listen. For three years.
I offer a slightly hacky solution that should work for all versions 4.0+ and into the forseeable future. If you use this solution and your PC blows up, I am totally not responsible.
Open Terminal (Applications > Utilities > Terminal.app).
Type the following command (as a single line):
sudo sed -i.bak s/Parallels_Desktop_Overlay_128/Parallels_Desktop_Overlay_000/g /Applications/Parallels\ Desktop.app/Contents/MacOS/prl_client_app
Enter your root/administrator password and wait for the command to complete running.
Congratulations; your Parallels icon should be a touch less ugly now. The above command creates a backup of the file that was changed, so if you would ever like to undo the changes, simply run (as a single line):
sudo mv /Applications/Parallels\ Desktop.app/Contents/MacOS/prl_client_app.bak /Applications/Parallels\ Desktop.app/Contents/MacOS/prl_client_app
I am still looking into whether a completely custom icon can be used (so parallels doesn’t switch to the windows-desktop-monitor icon), but haven’t been able to find the source of that one yet.
- virtualising windows
I primarily use OS X Lion on my laptop for menial tasks, as well as development for anything that isn’t .NET. For the .NET side of things, I prefer Visual Studio over other available options, and have therefore opted to keep a Windows 7 multi-boot setup. As per my previous post, this does come with some problems when I am trying to make my shared files accessible from both OSes.
Over the last few weeks I have been in an ongoing fight with various partition types, NTFS drivers for OS X (and HFS+ for windows) and am not yet happy with any of these options. Metadata for files is not correctly handled with any of the shared filesystem setups I tested, which leaves messy attribute/metadata files lying around in directories, synced with dropbox and thrown into other people’s shared folders.
With the recent release of Windows 8 dev preview, as well as major versions of the two competing virtualisation packages for OS X — VMWare Fusion 4 and Parallels 7 — I decided to nuke my dual-boot setup and give virtualisation another try.
I am currently in a state of having six virtual machine images across the two software packages (XP/Win7/Win8) in order to compare performance and stability. My journey still continues, but what I can say so far is:
- Neither option allow for hardware accelerated video in Windows 8. This seems to be an issue with driver signing/support. Windows 8 is quite chuggy without hardware accelerated drawing, so I don’t believe running it in a virtual machine is a great option for daily use or performance testing.
- VMWare is unstable. In several cases it would lock up at 100% CPU usage with no feedback on the VM.
- VMWare mouse integration with OS X is not perfect. When entering the VM window there is a 1-2 frame “jump”, where the cursor appears in the wrong position on both OSes before synchronising to the correct location.
- Neither option handles suspending in an optimal way. They both seem to save an image of the *full size* of the block of RAM assigned, rather than only the used portion. If you need quickly resuming VMs, you’re best off using the Windows hibernate options as opposed to the VM one.
I am planning on using Parallels for the time being, and testing how usable it is for day-to-day dev work in VS2010 (and VS11 preview). As I do have VMWare still setup, I may do some performance comparisons between them, but Parallels seems to be the better option this round.
Will report back with more detail once I have had some time to form a solid judgement.
- visual studio with multiple projects
When working with more than one project in a single solution, your working environment can get quite cluttered as you follow references between projects, sometimes with conflicting filenames (think generic things like Program.cs). This can lead to confusion as to where exactly the file you are working on is coming from.
I’d like to introduce anyone who is not yet aware of their existence to the Visual Studio Productivity Power Tools (released by Microsoft). These “Power Tools” are basically previews of new features and improvements which will likely be present in the next version of Visual Studio (vNext!), but Microsoft decided to release early to preview and get feedback.
I’d like to focus on one particular feature offered by this pack, which solves the project differentiation problem I mentioned above:
By enabling the Document Well 2010 option, you allow the PPTs to take over your document tabs and display them using a new (improved) format. The killer new feature here being per-project colour choices for tabs!
The defaults are a bit plain, but changing them only needs to be done once in order to get a really nice looking tab setup.
In this screenshot you can see I have files open from three different projects, grouped by project with clearly distinguishable colours. Also note that I have one tab which is pinned to the left — another useful feature of the PPTs — allowing it to always remain visible when many tabs are open.
The Document Well part of the PPTs has far more options to explore than the ones I have mentioned, including custom ordering, allowing multi-line display and scrolling and plenty more tick boxes for the control freak. I actually like the default behaviour of the VS2010 tabs for the most part, so only use the two particular features I have covered in this article.
Definitely worth checking out if you haven’t already, along with all the other goodies that come with the PPTs extensions.
- coding in colour
I spend quite a bit of time setting up my development environment to a perfect state. I define a perfect state as one where I can innovate, create and debug without thinking about — or being distracted by — my input/output environment. There are quite a few separate elements I’d like to focus on and share, but this time around let’s take a look at colours and syntax highlighting.
While I believe my colour scheme was originally based on another, I have unfortunately since lost its origins. It has been through many revisions over the years, but I have been using it for over a good length of time without any changes, so believe it is at a stable point to release!
You can download my visual studio 2010 colour settings here. Preferred colour schemes are quite a personal thing, so I’m interested in hearing about which parts people like/dislike about this scheme!
- local game dev meetup
Today I attended and presented at a local game development showcase event run by letsmakegames.org — “Show us your Bits”. I decided to focus on the process involved with porting osu! to the mobile platform, and the technical decisions and optimisations that consumed a large chunk of the development time.
I also had the chance to talk with more local game developers than I knew existed, and give some eager people a shot at osu!stream :).
The event was recorded so there will likely be a recording of my presentation (and others) available in the future, at which point I’ll make sure I link it. For the time being I’m going to dump my presentation’s slides here for public consumption. They may not completely stand-alone, but will give an idea of the stuff I covered. If you are interested in anything in particular, leave a comment and I can go into more detail!
Note that due to the filesize, videos are not included in this web-viewable copy.
- decommissioning a wordpress blog
One of my long-term goals has been to cut down on the number of servers under my control that require frequent maintenance and attention. For over a decade I have hosted personal content (including my blog) on a PC in my house, which has served many purposes over the year including streaming media, storage, web hosting and just as a general shell box for IRC and other persistent connections.
While hosting things locally can be great in some ways (full control, minimal latency), there comes a point when you realise you just don’t want to have the hassle of dealing with power and network failures that come with the household environment. Things also change over time, and most of the services provided by a local server — to myself as well as my family — are no longer being used; in fact the only remaining uses I have for it is hosting my personal sites and as a shell for IRC.
In moving my blog over to a hosted solution (tumblr, as you can see), I decided to use this as an opportunity to consolidate my hosting and make a step forward towards decommissioning my home server (falco).
The main two sites still being hosted on this server were my blog and splash page for ppy.sh. Today I will cover the wordpress side of things, and leave the other for next time.
My thoughts were to create an archived copy of my blog, keeping a fully navigable copy of the content without any security risks caused by maintaining a wordpress install or web server. A quick google search led me to a plugin called Really Static, which allows for static caching of pages. It has a manual option to create a static content of every page in one swoop, which was exactly what I was after.
After adding a “deprecated” header to the top of every page, I went ahead and ran the plugin (which took a good two hours due to my crippled apache2 configuration).
Apart from a few manual corrections for stray images (which could have been avoided with some better configuration choices in the Really Static settings), everything seemed to work perfectly. The static copy ended up being around 68mb of HTML and images.
I had been meaning to give NearlyFreeSpeech a chance for web hosting once I found a project small enough to do so, and this was the perfect opportunity. They offer hosting with a unique payment system where you deposit money and only pay for the storage and bandwidth you use. Kind of like cloud hosting without the running costs. A few calculations showed that with my expected traffic and storage requirements the costs would be well under a dollar a month. Couldn’t hope for any more than this!
So my old blog is already up and running on the new hosting in a completely static form. Go take a look if you are that way inclined (but don’t try to leave a comment on it; it won’t work :p).
- managing and repairing partition tables w/boot camp
It is no secret that I stand by Apple hardware — with my desk covered in far too much aluminium — but on a day-to-day basis I use OS X and Windows roughly equally. This means that for my portable setup (MBP 13” late-2009), I need a working copy of both operating systems, and the ability to switch between them with minimal effort.
Recently I have become more reliant on Dropbox in order to sync files between PCs (and in this case, OSes). This is generally more efficient than bringing git up-to-date each time i reboot, which would usually involve stashing, committing, pushing, pulling, popping stash every time. The problem created is that Dropbox doesn’t sync instantly — in fact it takes a fair amount of time.
Due to OS X not having write support to NTFS partitions (and vice-versa with Windows and HFS+) I had been unable to store dropbox in one location.
The plan? Create a new shared ExFAT (basically FAT64, with case-sensitivity and more goodness) partition which could be accessed by both operating systems, and use it solely to store my dropbox.
Intel 160gb SSD [120gb OS X | 8gb shared | 32gb Windows 7]
I have 11.2gb of dropbox storage using referrals and various promotions they have had over the years, but don’t plan on using this much space in the near future, so settled for 8gb. Disk Utility on OS X allows resizing HFS+ partitions on-the-fly, so I freed up enough space and tried to make this happen… and then ran into some problems…
It was not meant to be:
No go. Disk Utility gave me random errors while trying to use the space that I assumed to be free. A quick reboot into Windows made it apparent that Disk Utility was unaware of the Lion recovery partition — which exists after the end of the primary OS partition — and was allowing me to create a partition larger than I should in fact be able to.
Settling for around 7.3gb, I set things up and did a test run. Found that the default cluster allocation size in both OS X and Windows when formatting ExFAT is 32kb (probably as an optimisation for USB/flash drives, which are the primary target medium for the ExFAT filesystem). So I reformatted the partition, and decided to try my luck and also remove the recovery partition. It shouldn’t be used for anything under normal circumstances, right?!
It most definitely isn’t, but in the process I upset the balance of things. During the Boot Camp setup process, a hybrid boot sector is made on the target disk, containing both MBR(windows) and GPT(os x) records. By changing the partition layout in windows, this caused the two do become desynced. OS X would no longer boot, and on further inspection (using a Lion install DVD.. err, memory stick) thought the primary HFS+ partition was now ExFAT.
“Ohshi—” was the first thing that ran through my head, and I mentally calculated how much data I would lose should I need to restore from backups (around 4 days, mostly iTunes imports/tagging). But of course I couldn’t just give up and stop here; the night was young and I was curious as to how this could be fixed.
After doing a lot of searching around and finding some people in similar situations with the classic “reinstall your OS” responses, I set out with two utilities in hand that I was sure could be used to resolve the situation I caused.
rEFIt: an EFI boot manager and recovery suite, containing gptsync — an app specifically made to bring the MBR record back in sync with the GPT table.
gpt-fdisk (aka gdisk): a release of fdisk which has gpt support. handily available in binary form for every OS!
A quick test using the rEFIt gptsync tool proved unsuccessful; it tried to sync the GPT over to MBR but failed due to the changes I made to the MBR in windows. I couldn’t find a way to sync in the other direction. I took a photo of the disk’s layout at this point, which showed start and end sectors for each partition in both MBR and GPT layouts. This would prove a life-saver after I proceeded to destroy my partition layouts beyond perceivably recoverable levels (let’s skip this part lest this post turn into a book).
On to gdisk (running from windows), and I could now fix the gpt layout based on the photo I took previously. This process was fairly straightforward as I am familiar with fdisk commands, but there is plenty of information here that would help those that aren’t.
End result? After about six hours I had recovered both OSes, removed the Lion recovery partition and created my new shared home for Dropbox. Was it worth it? I’ll leave that for you to answer. I do recommend that if at all possible, you should do all your partitioning in OS X (and leave the recovery partition be) to avoid a similar situation.
I have been using a shared dropbox across both operating systems for a couple of days now and have hit a few bumps (which I will detail in a future post), but overall it feels *that* much better to not have to wait 10-30 minutes for Dropbox syncs. I have been switching OSes a lot more since the change, which means overall more productivity.
Lessons learnt:
- Backup before playing with partitions.
- Always keep a record of the original partition table layout, so you can restore if something goes wrong. As long as you don’t format any partition changes all data is recoverable.
- Windows prefers reading and writing the MBR when available, even though it supports GPT.
- Keep rEFIt on a recovery USB stick or DVD (my 8gb usb stick now has OS X Lion Install and rEFIt).
- a personal challenge
Following a twitter conversation with a good friend of mine, I have decided to attempt one-blog-a-day until I run out of interesting things to post about.
My aim is to post on topics that can’t fit in a tweet, while not impeding on my schedule to the point where I can’t make a daily post. As with all blogging experiments, I’m not sure how long this will last, but you can’t do more than start positively and hope for the best.
I do have a good deal of things to post about saved up over the years, but not recorded anywhere, so this will be an impulsive process mostly focusing on what I am currently working on/with.
This will deprecate my previous blog (which can be found at pe.ppy.sh). I considered migrating content, but after weighing up the pros and cons decided to leave it be and start fresh.
It shouldn’t need to be said that the majority of the posts will be of technical nature, so if this frightens you, run away now. Run for the hills.
update: looks as if another friend will be participating in this challenge too!
-
Photos
-
Tweets
- having a human readable email address saves a lot of pain on low-quality phone calls. pe@ppy.sh takes about 5 confirmations in most cases.
-
Repositories
- 1 fork / 1 watcher / Pushed Nov 13, 11
- Port of the SGL for .NET 2.0 and an osu! storyboard code editor1 watcher / Pushed Sep 17, 11
-
Uploads
-
Recent Tracks
- The Bottom Feeders by Darren Korb
- Single Ladies - Original Mix by Neuron Attack
- また、近いうちに by チャットモンチー
- My Sugar View by チャットモンチー
- セカンドプレゼント by チャットモンチー
- あいかわらず by チャットモンチー
- あいかわらず by チャットモンチー
- 雲走る by チャットモンチー
- 青春の一番札所 by チャットモンチー
- キャラメルプリン by チャットモンチー
PerfumeチャットモンチーsoundTeMPMasayoshi Minoshima소녀시대NhatoLixDavid GuettaCapsulePax Japonica Groove綾倉盟SAISEN TURN feat. Big ArrowBenji GoodrichASIAN KUNG-FU GENERATION久石譲fu_mou坂上なちSOU1Big ArrowLiaZeddLondon Elektricitymican*Aira MitsukiNeuron AttackDaito MusicきゃりーぱみゅぱみゅAbove & Beyond feat. Richard BedfordManami MoritaHalozyKlaypexYUIDiverse SystemM-NekoMisao Aoyama坂本真綾AyameMarcus SchössowほたるProject SyncAqua上戸彩Alstroemeria Records来兎Darren Korb古川未鈴Nhato vs HedonistYUKIFace 2 fAKENhato winwiki 綾倉盟- 43 plays
- 25 plays
- 24 plays
- 23 plays
- 22 plays
- 21 plays
- 20 plays
- 19 plays
- 18 plays
- 17 plays
- 17 plays
- 16 plays
- 16 plays
- 15 plays
- 14 plays
- 14 plays
- 14 plays
- 14 plays
- 14 plays
- 14 plays
- 13 plays
- 13 plays
- 12 plays
- 11 plays
- 11 plays
- 11 plays
- Lunar Dial by mican*11 plays
- 10 plays
- 10 plays
- 10 plays
- 10 plays
- 10 plays
