My I-also-think-Apple-is-sauced post

I've been following the total shit storm blowing all over the web re: the new iPhone OS 4 ToS and the dreaded clause 3.3.1 from my favorite perch at HN. Been mulling a post since this whole thing started and all of a sudden posterous gives me a reason to do it. Everyone who knows me knows that I am totally into Apple and their products. Unfortunately, I love their Products but hate their Policies. For those not in the know lets take a look at the latest wrinkle in the Emperor's finest robe:

3.3.1 — Applications may only use Documented APIs in the manner prescribed by Apple and must not use or call any private APIs. Applications must be originally written in Objective-C, C, C++, or JavaScript as executed by the iPhone OS WebKit engine, and only code written in C, C++, and Objective-C may compile and directly link against the Documented APIs (e.g., Applications that link to Documented APIs through an intermediary translation or compatibility layer or tool are prohibited).

Oy, where to begin... Ok, lets start with some programming 101. Virtually everything that you and I do on a modern computer is written in a language the computer does not understand. Programmers write programs in many, many, many, many different languages. Almost too many to count. Simply, what happens next is that these programs are translated into machine code that the physical computer actually knows what to do with. Further, programmers are know to program in one language to translate into another and so on for a long time now. The fact that Apple has unanimously decided to usurp the practice of programming for it's product is, simply, outrageous. On its face this ToS change is ridiculous, counter productive and a step backwards for the discipline. Also, I plainly do not believe it is enforceable at a technical level.

Most everybody who has even a passing interest in the iPhone knows that the only way to get a program onto an iPhone is via the prescribed anointed channel - The App Store. What is less known is that there are other ways to get illicit (from Apple's perspective) software on the iPhone by means of - what boils down to - exploiting software vulnerabilities in the iPhone OS.  Apple succeeded in creating a closed ecosystem and justified it under the guise of securing their product on behalf of the users (Oh, the children! Save the children!). And now Apple is taking another step on the other side of the device by dictating more stringent terms to their developer community. Even before this latest encumberment, there was a lively discussion revolving around the seemingly capricious position Apple was taking by acting as judge, jury and executioner for all who wanted to deploy on iPhone OS. With this latest step I am here to tell you that Apple has officially crossed the line. 

Lets talk in practicalities. Like everyone else, programmers have areas that they excel in. In particular programmers gravitate towards different languages for one reason or another and make their home in one or more language. By limiting which languages are acceptable, Apple is forcing those who want to program for the iPhone to become proficient in one of their anointed languages. According to these new rules, third party tools which allow a programmer to program in another fashion and produce output which will work on the iPhone OS are verboten. Unfortunately for Apple the world is a lot smaller than it was in the 1980's and 1990's. Word spreads, the few people who actually read ToS's tweet their findings from their android mobiles. Perhaps most corrosively, cultures evolve. The developer community today is not the same as decades past. Today's best and brightest are milling about the Google Code, Bitbucket and Github water coolers. Coding has moved from a solitary pursuit to a social engagement. Coders and hackers and programmers are actively seeking each other out and discussing the latest and greatest events of the day in real-time, all the time. This move seriously threatens to undermine Apple's credibility in this community. The very same community that can make or break the iPhone OS in all it's hardware manifestations.

At the end of the day, this rule change is not only bad for programmers, bad for consumers but most importantly it sets a terrible precedent for the future of programming. Where is the freedom to innovate when all directives come from a central command? Recent history has already shown us how IBM and Microsoft abused their position to their long term detriment and how a command economy can fail. My related comment on one of the HN threads regarding this issue had an interesting historical reply. Turns out AT&T tried a similar thing back when they ruled the roost. Imagine that you had to use one of AT&T's phone to make a phone call? Absurd? Not so. Ironic then that Steve and Woz got their start hacking hardware to scam long distance phone calls from AT&T! Meet the new boss, same as the old boss. I really want to see Apple win in the marketplace. They have been for a while now and I would like to see it continue - but at what price? I'll reserve judgement till iPhone OS 4 actually ships but developers have to agree to the terms now. I, for one, say that if things stay the way they are it is too high a price to pay and I'll just shift my stack and double down on Google.

Sad face - "Too many open files"

While developing a new project that utilizes node.js as the webserver and riak as the datastore, I encountered this error (here is a relevant dump):

InnoDB: Operating system error number 24 in a file operation.

InnoDB: Error number 24 means 'Too many open files'.

InnoDB: Check InnoDB website for details

My developer platform is a beefy mac with lots of ram that lets me run many virtual machines at the same time, but I digress. Thankfully, riak dumps operations to a log file in its log folder so basically all you need to do to get that info is to check your most recent log like so:

$cat < `ls -tr log/erlang.log.* | tail -n 1`

The problem has to do with the fact that the Mac (and other *nix's) limits the maximum number of files you can have open at any one time. Unfortunately, as my testing has shown, riak likes to hog files. Especially when using the innostore backend. More or less, as of this writing (riak v0.8), the innostore backend to riak will open 64 files for every bucket that you use. Even if there is no data in the bucket. A call to "http://host:port/riak/someBucket" will instruct innostore to create the 64 files if they do not exist and open them. Each file is about 100KB with no real data in them, aka empty, in case you were wondering. 

So how do I keep testing while Basho figures out a way to fix this issue? By upping my max file limit, of course! First let us check what are current limits are:

$launchctl limit

Note that that limit is a user specific limit. The kernel has its own limit which is check like so:

$sysctl -A | grep kern.maxfiles

I got most of this information from krypted.com, which has the most cohesive description in one place that I have yet to find on this subject outside of kernel-developer mailing lists. I got the userland shell info from here. Now for the eagle eyed, those two pages may have some overlapping yet conflicting information. I employed the following method, which may not be best for all users, YMMV. Be warned.

Borrowing heavily from the aforementioned posts, I simply created an /etc/launchd.conf file like so:

$sudo touch /etc/launchd.conf

Then edit the launchd.conf file to include these lines:

limit maxproc 1024 2048

limit maxfiles 2048 4096

The two columns are a soft limit and a hard limit. The distinction being... eh whatever, google is your friend. But importantly, the soft limit must be a number smaller than the hard limit and the hard limit may be "unlimited". I also added a maxproc line in there for good measure while I was at it. Of note, after reboot, sysctl and launchctl command both return the new values. If I didn't have so many windows open right now I would do more testing about that, but eh, whatever.

[UPDATE 10MAR2010]

Got an email from Jon Meredith, a Basho employee stating the following:

You can control the maximum number of file descriptors that Embedded
InnoDB will use by setting open_files in the innostore configuration.
The default value compiled in is 300 which exceeds the OS X default of
256.

In riak/etc/app.config add this to your innostore section

{innostore, [
           %% Other Inno settings
           {open_files,               100}            %% Limit to 100 open file handles
          ]}

So there you have it. A number of ways to work around the open files limit on macs.

Lets admit the iPad is an iLetDown

I wouldn't be true to all the people out there who know me as an Apple super user and always ask me which Apple products to get if I didn't tell it like it is and call Apple out when they iFlop. I'm a big Apple fan, have all kinds of Apple products from the Mac Pro to the iPod shuffle. But I gotta say I feel kinda let down after this mornings announcement. Lets take a look at whats iWanting (in no particular order):

  • no background apps
  • no multitasking
  • no osx
  • no root
  • no camera
  • itsy bitsy baby storage
  • no removable media
  • no GPS
  • 3G from... ATT?
Lets delve into these points one at a time:

No Background Application
One of the biggest drawbacks of the iPhone OS, which powers the iPhone and the iTouch and soon to be released iPad, is that regular non-Apple applications can not run in the background. Basically when you open that nice shiny Facebook app and then you wan to turn to your even shinier weather app, Outside, your iPhone basically closes Facebook and opens Outside. Now this might be all well and good for some apps, but what about others that need to maintain a connection to the mother ship, like say, an instant messaging app? Or an app that does some processing for you? Well, your'e kinda out of luck.

No Multitasking
See above. Want to switch between apps or a number of apps? They all need to save their state on exit and reinitialize when they open up. Slow, slow, slow.

No OSX
iPhone OS is not OSX. It is a neutered derivative. Want to do all the interesting things you can do on your Macbook or iMac or Mac Pro? You can't. Like finder. Want to share files between applications? No, can't do that. 

No root
Now this is a little technical but on unix based systems (of which all OSX systems are based) "root" is what's known as the master user. This user can control what happens within the operating system, more specifically has privileges other users do not. This concept is not exposed to the iPhone OS. This is all well and good (to a degree) on personal devices like the iPhone or iTouch. But the iPad strikes me as a multiuser device. I haven't seen anything yet that speaks to this fact. How do you share an iPad in a household without letting all the users have access to everything on the device?

No Camera
This is probably one of the biggest blunders. Virtually every consumer electronics device that plays in the computer space comes with a camera nowadays. In fact, I believe the iPad now follows the iTouch, iPod classic and iPod shuffle as the only Apple devices that come sans camera. Interested in new augmented reality applications? Ain't gonna happen. Video conferencing from the couch? Nope. Pictures of your cat sitting on your lap while reading the NYT? Not bloody likely.

Itsy bitsy baby storage
64GB max internal storage? Give me a break. Steve, et al want this device to be your library, your jukebox and your vcr? And they only give you 64GB? Full stop.

No removable media
See above. The newer Macbook Pro's now come with a builtin sd card reader. Those little sd cards are virtually ubiquitous when it comes to removable storage. You can find them in everything from digital cameras and HD video camcorders to portable gps devices which keep a running record of your last 50 mile bike ride. Which segues nicely to:

No GPS
What's the matter, Apple? No love for Foursquare? No love for location based applications like navigation or search? So the accelerometer and compass are just for games and knowing which way to look for the sunrise after a long night reading ebooks.

3G from... ATT?
What the deuce? This just smacks of "we had no time to line up other providers so we went with the junk that we already know." And an extra $130 for the 3G chipset. Come on. And the $15/month plan for 250MB? Come the #@%^ on. How bout you give me 10GB from your "unlimited" plan for $15 or $20 and call it a day.

 

My gripes fall along two main areas, software and hardware. I get the argument that background apps and multitasking drain battery life. But this is not a pocket device. This is something that borders a laptop and as such needs more software horsepower at the OS level than just "lite" applications we all call "apps." As for the hardware, the surface area alone of this device is something like 7 times that of the iTouch. Cram some more goodies in there, will you? Have you seen the Ipod Nano? The thing is miniscule and comes with a video camera! Apple could have put two cameras on this thing - front and back. Not to mention an sd card reader, and a gps chipset.

Lets face it, this is a larger iPhone 3Gs or iTouch with no calling plan. And the name is iLame, as a friend of a friend said, "sounds like a feminine hygiene product, ipad maxi for the large model." Will the iPad find a market? Sure. Will Baby boomers appreciate the larger fonts and soft keyboard? I know a few who certainly will. Will the Publishing industry sacrifice their first born children to get their content on this device? Absolutely. Will the ebook experience be "insanely great" as Jobs would have you believe? Probably. Will I recommend it to my friends? Probably not. Get the iTouch or save your shekels for a Macbook. If you just want an ebook reader, get the Kindle. Better yet, just don't buy ebooks that are laden with DRM.

At the end of the day this can only be seen as an evolutionary advance over the iPhone and iTouch, if that, and not the revolutionary device I know I had been hoping for. File this one under iFail. What do you think? Does Apple have another winner or will the iPad be sitting next to AppleTV getting drunk at next years Apple Christmas party?