2019 Update: I did not receive approval from my employer to continue developiong this app.
2021 update: I deprecated this app in favor of working on other projects!
One of my favorite activities while traveling is going on a local walking tour, mostly because I get the chance to check out the architecture through the eyes of a local. One of the pain points I always had, however, was that I would spot an interesting construction that wasn't a part of the tour, and I would never get the opportunity to learn more about it. Great design, lost to the ether! After commiserating with fellow travelers who had passing interests in architecture, I thought I'd try and alleviate the problem.
The infinity loop (borrowed from my days @ IBM) envisions product development as a continuous improvement model. The end user is placed in the middle of a research and dev loop where each loop is validated with a series of playbacks, or user interviews. As the sole developer, I knew it was crucial that I avoid costly dev missteps likely to sink motivation.
Personas I formalized my conversations with other travelers into high level personas. These could have used a much more rigorous interview process, as well as other methodologies, but observational would have been pretty creepy and literature/ESM is non-existent. I'll note up front these were mostly sourced from my personal contacts, so obvious demographic biases are present.
Pain Point Validation Pain points differed across business and leisure travelers, with business users bemoaning the lack of time. They would be in on a Tuesday and out on a Thursday, passing by interesting buildings on the street with no time to learn more about them. Leisure travelers loved guided tours on architecture but also found them expensive and increasingly outcompeted by travel bloggers or free self-guided tours. Interestingly enough, our Cultured Cranstons were truly living the dream with narry a worry. Expensive travel packages with reknowned architects as guides provided a full range of services. Costs were not a serious concern; the tour company traveled in groups of 20-25 so the social component was present; time was booked and planned well in advance; the personal, expert touch was there as well.
Market and Competition Validating the market for each of these personas and pain points was a fun exercise, especially since this was a passion project with no plans for monetization. The standard market analysis (TAM -> SAM -> Competitive Landscape -> Segmentation -> Strategic Differentiation -> GTM) seemed slightly overkill, but I did end up putting some boundaries around each of those areas. Ultimately the insight around unit spend and mobile user behavior helped crystallize a GTM direction, so I dive deep on these below.
Combining persona, market, and competitive insight led to the following high level features:
| # | Feature | Priority | Insight or Pain Point |
|---|---|---|---|
| 1 | Native iOS app | p1 | Mobile device penetration for travelers (customer acquisition) |
| 2 | Take photo of building using built-in camera | p1 | Alleviates customer pain point of missed opportunities to learn more about a building |
| 3 | Classify building features | p1 | Alleviates customer pain point of missed opportunities to learn more about a building |
| 4 | Info on architect | p1 | Differentiation: replaces "SME" knowledge found in paid options |
| 5 | Location-based push notification based on buildings in vicinity | p2 | Differentiation: personalization of content as value add over existing products |
| 6 | Model features in 3D with digital architect | p2 | Differentiation: replaces "guided tour" aspect found in paid options |
| 7 | Native Android app | p2 | Mobile device penetration for travelers (customer acquisition) |
| 8 | Historical context for architectural style | p2 | Differentiation: replaces "SME" knowledge found in paid options |
| 9 | Save and share previously classified buildings | p3 | Social network boosts stickiness and alleviates customer pain point of not having enough time to learn more about these buildings |
| 10 | Offline mode | p3 | Differentiation: accessible anywhere the user is located |
The initial cut of features was the ideal experience (the dev side of me was already cringing) and directly addressed pain points across Abigail and Bobby personas, as well as captured some of the mobile behavior insight gained via research. From a competitive standpoint, these features mostly sought to replicate the guided tour value prop while remaining free to the end user. From a GTM standpoint, I was pretty firm on an iOS-only rollout due to CAC, dev experience, and potential trademark/IP issues (and prioritized accordingly).
Scope Validation With a healthy set of features it was time to make cuts for MVP; I normally do this by presenting the ideal experience via wires and have the devs spike out and iterate from there. Since I was wearing both hats this time around, I just shortened this to one loop and went from ideal experience to pixel perfect on the MVP.
Of the tradeoffs I had to make, I relied mostly on effort v. value. For instance I assumed heading into the project that publically available datasets on pre-modern architecture were of high quality. Once I actually dived deep into those DBs I quickly realized the level of curation was nowhere near robust enough. I also had to strike probably the biggest value add from a product perspective--surface detection and object recognition on the buildings themselves. Turns out the literature indicates only a .6 F1 score on a custom classifier trained against building facades. While I would have loved to have invested 5 months on a custom CV project, I was unfortunately stuck with choosing open source models across existing enterprise companies.
Ultimately, Microsoft's GUI for custom training won the day. They also have the ability to package the runtime into a 2.9 MB executable that was essentially a drag-and-click over to XCode for iOS development.
The last major feature I decided to cut was the sharing feature. While great for building stickiness, it required me to break the seal of localization. I would also have had to host the user profiles and metadata, something I'm fundamentally averse to from a privacy perspective.
So I played back the MVP userflow to a few friends and resoundingly the conclusion was: "this is cool, but I probably wouldn't pay for it." Given the lukewarm response, I decided to shift the GTM OKR I planned to track from acquisition to retention. I figured if I couldn't deliver the killer app I could at least deliver one that kept people around longer than average. Dreams of a groundbreaking travel app: finished.
Oh well, there's always v2!
Onramp and First Cut My last legit iOS project was a few years old, so I came limping back into XCode and validated the architecture in the meantime.
Development ended up being less onerous than I thought--after brushing up on XCode basics I was prototype ready in about 3 weeks. After the MVP cuts I ended up scratching the left hand side of the architecture doc, and I found open source code everywhere for both the CameraViewController and the ARViewController.
Playing it back A ton of insight came out of this playback cycle with my travel users, mostly because the last time they had seen this it was in Adobe XD wires, which never seems to command the user's focus like physically navigating the application does. The feedback was loosely collected into three buckets:
The classification piece was the biggest and the obvious impairment to this being a killer app. I had already accepted I wasn't going to write a custom building facade classifier though from an effort perspective. Nor did existing databases support curating individual buildings. The second and third pieces were more interesting. Folks spent on average 45 seconds before successfully getting to insight in the second page. I knew this would be devastating in a real-life scenario and likely lead to poor retention. And since I had removed a user login, it seemed users were expecting some sort of splash screen or home page.
The "Architect" Problem Placing the 3D model of a representative building and learning more about its distinguishing features was a key facet to learning more about architectural styles. Knowing users were having problems reaching insight meant I needed to introduce info quicker. However, now the issue was how to deliver a good chunk of info. I briefly considered having a 3D architect person in the background with a speech bubble and/or audio, but quickly discarded that due to accessibility issues. Next I considered placing 2D text on the model itself, which turned out to be quite a rabbit hole.
To get large, text-based chunks of info across to the user in ARKit there are essentially 3 options: layering 2D text on 3D objects, layering 2D text on a separate plane within the environment, or creating 3D text directly.
In the model above, this would be the difference between having the text on the plane behind on the model, on the model itself, or just regular old 3D text in the environment.
I tried the embedded method initially, but I encountered two issues. One, the user's eyes would track immediately to the plane and start reading instead of viewing the model. Two, it turns out I could distill the amount of text needed to just a few words per model.
Plane-based text was also eliminated because it needed the following code:
let plane = SCNPlane(width: 20, height: 20)
let material = SCNMaterial()
material.isDoubleSided = true
material.diffuse.contents = skScene
plane.materials = [material]
let node = SCNNode(geometry: plane)
scene.rootNode.addChildNode(node)
This would require generating tuples of the model and the plane itself, with corresponding anchors. The hit testing in ARKit is pretty robust, but roughly 1 in 10 times, especially in low light scenarios, I'd find the anchors collapsing and the models skating off into high heaven or other strange behavior. It presented enough that I decided to scrap it as well.
The final piece of dev action was creating a splash screen on "first download" to guide the user.
Final playback and GTM I had my travel-user-friends in TestFlight for the final set of playbacks. Nothing crazy came out of this as the functionality didn't change significantly from the last session. Last thing I tweaked was the intro screen that helped people with the navigation. I turned it from text-based into logo-based. Then, it was launch day!
Apple really has new app release down to a science. Turnaround time was under 48 hours from cutting the release branch to going live on the store, and they provide a really robust set of analytics from day one.
From an OKR perspective, at roughly 1.5 months, there's lots to experiment with!
Conversion when tracked against product page views (when someone clicks on the actual app) is 20% (500 page views -> 100 downloads). But conversion tracked against impressions (when the app comes up in a search) is less than 6%. So Apple's built-in marketing tools could be an option near the next release cycle.
Retention is a different beast entirely as there's not enough data to calculate drop off. Looking at DAU (2-5% of total installs) alone, I'll likely struggle with the same severe dropoff as most other mobile apps. I'll check back in around the 3 month mark once the numbers are up.
Conclusion: okay-ish first attempt with levers yet to pull on conversion. The key will be delivering v2 with an updated CV model that can detect individual features.