Offline First Web Development

4 (1 reviews total)
By Daniel Sauble
  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. The Pain of Being Offline

About this book

When building mobile apps, it’s easy to forget about the moments when your users lack a good Internet connection. Put your phone in airplane mode, open a few popular apps, and you’ll quickly see how they handle being offline. From Twitter to Pinterest to Apple Maps, some apps might handle being offline better—but very few do it well. A poor offline experience will result in frustrated users who will abandon your app, or worse, turn to your competitor’s apps!

Expert or novice, this book will teach you everything you need to know about designing and building a rigorous offline app experience. By putting the offline experience first, you’ll have a solid foundation to build upon, avoiding the unnecessary stress and frustration of trying to retrofit offline capabilities into your finished app. This basic principle, designing for the worst-case scenario, could save you countless hours of wasted effort.

Publication date:
November 2015
Publisher
Packt
Pages
316
ISBN
9781785884573

 

Chapter 1. The Pain of Being Offline

Susan Hopkins is the VP of design at a tech start-up in Silicon Valley. At 63 employees and counting, her day is as much about hiring and culture as it is about design. In her 15-year career, she's seen plenty of both.

Right now, she's shepherding the design of the company's first 1.0 product. There are a lot of moving parts shared among product, design, development, marketing, sales, documentation, and support. She has to wear a lot of hats. She loves every minute of it, but it's a challenging job.

Fortunately, Susan has to-do lists. Everything in her life is organized in this way: shopping, work, personal projects, gift ideas, movies to watch, books to read, travel destinations, and more. As long as she can create and check off items from her lists, life is good.

Unfortunately, not all is well today. Her favorite to-do app, the place where she stores all of her lists, behaves poorly when offline. When the Internet connection goes down or becomes unreliable, Susan has the following problems:

  • Her to-do lists fail to load.

  • She doesn't have confidence that the to-do items she created on other devices are being shown on this one.

  • The source of the truth is in the cloud. When she is offline, her devices tend to diverge from this source in reconcilable ways.

  • Susan attaches photos and videos to her to-do items. Any data connection slower than 4G is an exercise in patience.

  • She has been an evangelist for this app, which has gotten some of her friends to use it. They like sharing items as a group, but when their phones are offline, they can't.

  • The app has a tendency to show scary error messages. They make her think that she's lost some data. Sometimes she has, but it's never easy to tell.

Susan closed her laptop lid. The conference's Wi-Fi was really bad this year. Susan wasn't the kind to give up easily, but you can't be productive on an unreliable Internet connection. Fifteen disconnections in one hour? Really? A few seconds ago, her to-do app crashed, losing the last fifteen minutes of work. Just great!

She opened her phone. No Signal showed prominently in the upper left corner. No luck there either. Bother.

This seems to happen all the time lately. Her rural home is in a dead zone. Her commuter bus spends twenty minutes out of every hour in tunnels and behind mountain ridges. She encounters infamously bad Wi-Fi at every other Airbnb accommodation. Even at work, her Internet connection seems to go down once a week—and always at the worst times.

It wouldn't be so bad if my applications handled these situations gracefully, she thought; but they don't. They crash, and stutter, and freeze, and lose my data, and raise my blood pressure.

To be fair, not every application is that bad. E-mail is generally solid, e-readers have all the content that I want, already cached, and my office suite doesn't care at all. However, this to-do app drives me crazy, and it's not the only app that is unreliable. Social apps, mapping apps, audio streaming apps—they all have the same problems.

A thought occurred to her.

What if a proper offline experience was the starting point? What if developers made offline functionality their top priority instead of the first feature to get cut? What if we lived in a world that was optimistic about fast, reliable, and ubiquitous Internet connectivity but didn't assume its existence? People would find their apps more useful and have less frustration in their day-to-day tasks.

Phone and laptop forgotten for now, Susan pulled out her sketch pad and a black marker pen. She started drawing.

 

The offline paradigm


Internet connectivity is only one part of the puzzle. Modern devices rely on different types of wireless networks: GPS, LTE, 4G, 3G, Edge, GPRS, Wi-Fi, Bluetooth, and RFID, among others. If any of these go down, she mused, certain bad things can happen:

  • If GPS goes down, it can mess up your fitness tracking resulting in erratic maps and false progress toward your fitness goals.

  • As your mobile connection degrades from 4G to GPRS, web browsing slows and eventually stops. Wi-Fi has the same problem.

  • Bluetooth is also important. It connects us to our keyboards, mouses, headsets, and other devices. As the signal weakens, these accessories become unreliable or cease functioning altogether.

  • NFC, while not ubiquitous today, certainly will be so in the future. We'll depend on it for all or most of our Point of Sale (POS) financial transactions. If it goes down, can we use other networks as backup?

Susan stopped writing and put down her pen. She gazed reflectively at the list. The paper lay on top of her laptop as though mocking the brick that lay underneath.

So, with these failure cases, how might we develop our apps differently? More importantly, is there a common set of principles that we can use in order to avoid reinventing the wheel each time? What makes an excellent offline experience?

Hand reached for pen and she began a second list:

Principles for a great offline experience are as follows:

  • Give me uninterrupted access to the content I care about.

  • Content is mutable. Don't let my online/offline status change that.

  • Error messages should not leave me guessing or unnecessarily worried.

  • Don't let me start something I can't finish.

  • An app should never contradict itself. If a conflict exists, be honest about it.

  • When the laws of physics prevail, choose breadth over depth when caching.

  • Empty states should tell me what to do next in a delightful way.

  • Don't make me remember what I was doing last. Remember for me.

  • Degrade gracefully as the network degrades. Don't be jarring.

  • Never purge the cache unless I demand it.

Susan put her pen down. A good list. She gave a contented nod, put the paper in her bag, and walked out of the café, humming to herself. It was an interesting thought exercise and possibly more. She would bring it up with her design team at their 10 o'clock review. Their project might benefit from these principles.

 

Developing for a worst-case scenario


A common practice is to design for the happy path, a scenario where everything works correctly and according to plan. These scenarios tend to paint an overly optimistic picture of the real world. When the happy path meets real-world problems, the charade is over and the app must undergo serious revision or a rewrite.

This is bad for a number of reasons. Obviously, rewriting an app is costly. One of the tenets of good software development is to fail quickly and rapidly. If you finish your app, only to find that it doesn't work as advertised, you've failed far too late. Even if you don't have to rewrite the app, bolting on fixes to address fundamental problems with the underlying architecture can cause bugs that tend to linger until the inevitable rewrite.

The alternative is to start by designing for a worst-case scenario. Worst-case scenarios are more representative of the real world so apps are more likely to behave as you expect. These scenarios tend to be harder to solve but result in a robust solution that covers more edge cases. As a worst-case scenario is a superset of the happy path, you get two solutions for the price of one.

This strategy can be employed in many other areas, such as security, responsive web design, accessibility, and performance. The approach is simple. Pick one or more worst-case scenarios that impact your app the most and build with these scenarios in mind. A worst-case scenario is an excellent constraint. The more constraints you add, the easier it is to design a great solution.

So, what is an offline user experience? We can boil down the principles from Susan's list in the previous section to form a concise definition:

An offline user experience bridges the gap between online and offline contexts by providing equivalent functionality and content to the extent that technology and the laws of physics will allow.

To design this experience, it's vital to understand the pain and frustration that people encounter every day. There are a million ways to build an offline-first experience, but the correct solution is formed by the needs of your audience. The story in the previous section was one example but you can find an infinite number of scenarios. Once you know who you're developing for and their pain points, devising a solution is much simpler.

Let's go through a few more offline scenarios to get you going. Once you start thinking about who your audience is, write scenarios such as these to describe how the offline experience impacts them. Talk to people. Observe them in the real world. Do research to see what others have found. These are all the things that can help you build a more accurate picture of your target audience.

Going on an off-the-grid vacation

Once a year, Carl disappears into the wilderness for a two-week trek along the Pacific Crest Trail in the Western United States. He does this to get away from technology. However, sometimes, due to the nature of his job, he has to respond to an important e-mail or take the occasional phone call.

There is no cellular coverage out in the pines. To connect, Carl has to leave the forests behind and hitch a ride to the nearest hotel. After a hot shower and some real food, he powers up his Android and endures shoddy hotel Wi-Fi for the evening. It's an inconvenience but a compromise that Carl is willing to accept.

Living in a third-world country

Marsha owns an old Nokia phone with an i9 keypad. She has access to a slow GPRS network, which often goes down. As modern websites are so bandwidth-intensive, her phone so slow, and online connectivity so tenuous, she turns off images and media when she browses.

In the next town, the mobile network is slightly better but it's a five-mile walk. When Marsha needs to e-mail photos, she composes draft e-mails and queues them on her phone. Once a week, she treks into town to deliver milk from their small herd of cattle. During this time, she is able to send those e-mails. It's a slow and frustrating experience but she can't do much about it.

Commuting on public transportation

Elizabeth travels on a bus for an hour each way on her daily commute. The bus doesn't have Wi-Fi and encounters several dead zones on the route. She is a dedicated bibliophile, so these routes are a great opportunity to feed her reading addiction. As she reads 1-3 books a week, she often runs out of reading material.

When this happens on the bus, she can't usually do anything about it. Her life is so busy that she mostly doesn't have time to think about downloading new books before her commute. Maybe she'll bring a paperback along next time for something to do.

Working on a Wi-Fi only device

Shawon is 14. On his last birthday, his parents bought him an iPod touch. He wanted an iPhone for the coolness factor, but, oh well. It's fine. It doesn't have GPS or a mobile plan but most of the time, it's as good as a real iPhone.

Like most kids, he relies on his parents for transportation and accompanies them on their errands. As he has a Wi-Fi only device, he often goes somewhere new, only to have to figure out what the Wi-Fi password is—if the place even has Wi-Fi. There aren't many Wi-Fi networks without passwords these days.

When he can't find the password, he has to stay contented with the games and music that are on his device. However, most of his games require an Internet connection, which is odd, particularly for games with a single-player mode.

Sharing files between mobile devices

Francine likes her world where file sharing is so easy and seamless. In 2005, you used USB drives. In 2015, you share files wirelessly, with technology and services such as Dropbox, Google Drive, and Airdrop. Unfortunately, when her Internet connection dies, she's transported back to 2005.

Just last week, her editor asked for an update to an outline that she wrote. Traveling at the time, she didn't have easy access to Wi-Fi. She made the changes on her laptop but couldn't transfer the file to her phone in order to be e-mailed. After struggling with this for several minutes, her Airbnb host replied with instructions to connect to the network. Crisis averted.

Streaming a high-definition video from YouTube

Brian streams online videos, a lot. Most of this streaming is over his phone. His mobile provider is kind of horrible, so he has to deal with dodgy Internet connectivity. Most of the time, YouTube is very good about decreasing the quality of the videos to compensate for bad network conditions, but it isn't always enough.

He wishes there was a way to get a transcription of the video or just wait for the entire thing to download before playing it. YouTube won't download an entire video when it's paused. Unfortunately, these features don't exist. Instead, Brian bookmarks the videos to watch at home.

Online shopping with your phone

Jasmine is not an organized person. She likes taking care of things in the moment, as they come to mind. As a busy professional, she doesn't have the mental resources to memorize lists and doesn't want to be tied to a to-do app or sheet of paper.

Shopping is one example. Throughout her day, she does a mental inventory as she moves about. When she spots an item in short supply, she grabs her phone and places an order with her Amazon app. When she's connected to the Internet, this works great. When she's not, it frustrates her that she can't take care of things. If only Amazon provided a better add-to-cart experience for people who want to buy things while offline.

Getting work done at a conference

Conferences are notorious for their abysmal Wi-Fi. Raphael is attending Whole Design, along with 1,500 other attendees. It's a three-day, single-track conference. On day two, with four hours to go, he is ready to be done fighting for bandwidth. He hasn't gotten much done beyond seeing some great talks. Even his cell phone, normally reliable, doesn't work here at all.

Every couple of hours, the attendees get a 30-minute break. Raphael uses these opportunities to sprint outside the venue. Here, he can get a solid 3G connection. This is enough to check his e-mail and chats and generally check in with his world. At least, it should be. His apps behave as though there's a binary switch between no bandwidth at all and enough to stream an HD video. A third option would be nice.

 

Principles of a good offline design


Once you've compiled a few scenarios such as these, identify some common principles to build your app around. Here are the principles that Susan identified at the beginning of the chapter, with a more detailed explanation for each one.

Give me uninterrupted access to the content I care about.

When offline, you don't want to open your app and have no content to consume or interact with. You want to have lots of content available. As an extreme example, a web browser that allows you to surf the entire Internet, as it appeared when you were last online, would be a fantastic experience. This doesn't exist because the laws of physics prevent such an experience.

Naturally, you can't provide an infinite amount of content for people. Instead, provide as much content as possible without hurting the user experience in other ways (that is, don't steal all their bandwidth, don't fill up all the space on their device, don't let the performance suffer, and so on).

Content is mutable. Don't let my online/offline status change that.

One common practice is to make content read-only while offline. When online, you can edit, delete, or manipulate the content, but offline, the only thing that you can do is view it. People want to work with their data regardless, but you're telling them that they can't, usually for no good reason.

Instead, as much as possible, make online behavior match offline behavior. Once you've decided what people can do with their data, try to apply these decisions to all situations. You'll make their lives easier, and yours, as you only have to design and build a single interaction paradigm.

Error messages should not leave me guessing or unnecessarily worried.

A cryptic error message is a bad error message. Well-designed applications should be clear, using human language and a consistent vocabulary. Error messages should be no different. Unfortunately, error messages are often overlooked in the design process. When this happens, the application assumes two faces: a friendly face when everything is fine and a distressing face when things go wrong.

The effect can be jarring and incomprehensible to some people. When their world is falling apart, they need more reassurance and help from their tools, not less. Make sure that you think about the wording in the error messages and how errors present themselves visually.

Don't let me start something I can't finish.

While performing a task, the further I get, the more vested I become in the outcome. For example, let's say I start a task consisting of eight steps. On step seven, if you tell me that I can't finish because I'm offline, I'll be frustrated. If you tell me that I can't save my progress and will have to try again later, I'll be furious.

Don't do that. Be up front about what I can or cannot do. Don't let me proceed down a path with a known dead end.

An app should never contradict itself. If a conflict exists, be honest about it.

In Chapter 6, Be Eventually Consistent we'll discuss split-brain scenarios. A split-brain scenario occurs when two people change the same data at the same time, on different devices. When the devices try to reconcile their data automatically, they can't. It's up to people to decide what change to keep and what to throw away.

In a poorly designed application, several things can happen. The app can clobber one of the changes without asking, resulting in confusion. It can show both of the changes side by side without explanation, which is also confusing. It might also enter a cryptic failure state, which leaves people even more confused and powerless.

Always get input from people when a conflict cannot be automatically resolved. If you must show inconsistent data, always provide an explanation.

When the laws of physics prevail, choose breadth over depth when caching.

As mentioned in the first principle, infinite caching violates the laws of physics. There will often be more data than you can reasonably cache. You have to make a prioritization decision. What data is most valuable to people? Cache that data first.

In most applications, there are different types of data that can be cached. As older data is usually less valuable than newer data, the more you cache, the value of caching in a particular category goes down. At some point, continuing to cache a particular data type is less valuable than switching to a different data type.

First, cache a small set of data across all the types. Then, go progressively deeper across all the data types simultaneously, with the more valuable types prioritized. As a result, no empty screens will appear when people skim the app. The UI will feel data rich on the surface and deep in all the right parts, which is what they want.

Empty states should tell me what to do next in a delightful way.

If there is nothing in the cache, you will have to show an empty state. There is very little that people can do with these states, which is a bad experience.

However, these states can be more than blank screens. They can tell people what to do next. They can be delightful: providing a clever message, image, animation, or video that people wouldn't ordinarily see. They can contain sample data, which people can interact with to see how the screen would ordinarily work. They can even keep people entertained with a game, puzzle, or other distractions.

Empty states are very easy to ignore. When engineering resources are tight, they are among the first things to be cut. Figure out how often people will see these relative to the other screens and then apply resources appropriately. During offline usage, empty states are more likely to occur.

Don't make me remember what I was doing last. Remember for me.

Context is king. If I close my app, preserve my current state. When I reopen the app, I want to see what I was doing last or find it simple to get there. The more complicated the app, the more important this is.

Many apps forget the context when switching from online to offline. They give themselves permission to reset certain parts of the UI, which is jarring and easily avoidable. As with most of these other principles, don't compromise on the attributes of a good experience just because an app happens to be offline.

Degrade gracefully as the network degrades. Don't be jarring.

Networks are not binary. There is a lot of gray area between not working at all and working very well. One of the goals of an invisible design is to avoid speed bumps in the user experience. An interface may switch abruptly from an online mode to an offline one, or worse, toggle rapidly between the two states. If implemented poorly, this will interrupt the flow and make it more difficult for people to get their work done.

Help the application see or anticipate an impending loss of connectivity. When appropriate, it should scale back its bandwidth usage, avoid massive changes to the interface, and gently nudge people if they are encountering any problems. The goal is a smooth transition from online to offline that informs the user in an unobtrusive way.

Never purge the cache unless I demand it.

Cache is the most valuable aspect of an offline experience. If the cache goes away while offline, people won't have any content. This is a bad experience. Protect the cache at all costs. Don't clear it unless people ask for it explicitly (usually for security reasons).

The latter implies an affordance to clear the cache. Web browsers have one but a lot of apps don't. This is extremely important, particularly when login credentials are involved. People may want to purge their data for a good many reasons. They may even need to do it remotely from another device.

 

Making the case to yourself


Unreliable and nonexistent Internet connectivity is a widespread problem, both in highly developed nations and nations that are less developed. Consider a few statistics from ITU, the United Nations specialized agency for information and communication technologies (http://www.itu.int/en/ITU-D/Statistics/Documents/facts/ICTFactsFigures2015.pdf):

  • As of 2015, 4 billion people from developing countries remain offline

  • Only 9.5% of the people in the least developed countries (LDCs) use the Internet

  • Only 29% of the people in rural areas have mobile coverage that is 3G or faster

  • Only 1 in 100 people in Africa have a fixed broadband subscription

If you optimize for a fast Internet connection, you are making an implicit decision about who you're designing for. If you don't want Internet connectivity to dictate who can use your app, design the offline experience first. Even if you're building an application that only Americans will use, consider that potentially 3 out of every 100 people are eking out a mere 56 kbps. Every megabyte has a large impact on their user experience and ISP bills.

This doesn't mean that you have to build an app that looks like Craigslist. There are intelligent ways to make an app scale its behavior based on the Internet connection available. You can even use location services to become aware about the potential dead zones in time to take preventative measures. We'll cover these strategies in Chapter 8, Networking While Offline.

Are you convinced that building with an offline-first mindset is a good idea? Great! Now, how do you sell this vision to the decision makers around you? Many activities have a negative impact on productivity in the short-term but a positive impact in the long-term: writing tests, conducting user research, including good comments in the source code, and so on. In the same way, switching to an offline-first paradigm can be perceived as an unnecessary distraction, even though the long-term benefits are clear. Let's address this now.

 

Making the case to others


If offline usage is so important, why don't we optimize for it? One reason is internal: it's hard to change our innate behaviors and habits. Developers are on the cutting edge and have difficulty imagining a world where computers are slow and Internet connections are flaky. Fortunately, there are tools to help us simulate that other world.

Even if we overcome this obstacle, there are still external hurdles to change:

  • We have to convince other developers that this will help us build better products, more efficiently

  • We have to convince other designers that this will improve the user experience

  • We have to convince our boss or others in upper-level management that this will increase our revenue

These are harder conversations to have but worthwhile. One of the goals of this book is to provide the evidence that you need to convince the developers, designers, and management in your organization. Offline-first development can improve your ability to deliver in all of these areas.

Convincing other developers

Developers care about efficiency and don't want to write more code than necessary. The extra care that goes into an offline-first application can, at a glance, seem to necessitate more code. You might get pushback that the team wants to build the easiest thing first and leave the offline functionality for later. This is a mistake. Instead, have a discussion about the architecture and tools that can ease the development effort.

In Chapter 4, Getting Online, we'll show how tools such as PouchDB, remotestorage.io, and Hoodie, custom built for offline use, can greatly reduce the amount of code that your team will need to write. These tools allow you to write good offline code, once. As applications are built, they gain complexity. This complexity makes them more difficult to change in the future. Using the right tools up front can save days or weeks of development time over the long run.

To make the case, show all the ways that an online app that is not optimized for offline usage can fail. The following chapters will talk about these situations in detail. Describe the pain that users experience while encountering these scenarios. If possible, take an existing app and write a task-based script that exercises these scenarios.

If your team is still skeptical, find people willing to try offline tasks on the app that you've built or a simple prototype. Invite your coworkers to attend. There's nothing like a firsthand observation of people struggling with a key part of your app to convert team members to your side.

Convincing other designers

Functional designers care about how people solve problems using the app. The trick is that there are many problems to be solved and you can't expect others to give offline problems the priority that they deserve. One way to measure priority is by the number of people affected by the problem. As offline problems are so widespread, you can make a strong case that every person using your app will benefit to some extent.

To start the discussion, build a task flow diagram for your application. This will help you see the scope of the app from a workflow perspective. In the diagram, point out the offline error cases and how these will affect the experience that people have. When functional designers see all the dead ends that their target audience might experience, they will be motivated to join you.

Visual designers may be a bit harder to convince. They too care about the experience of an app but care more about the clarity of the presentation rather than the functionality. Take the task flow diagram that you built earlier and mock rough screens that correlate to each step in the flow, including the error cases. Make sure that everyone is aware of the total surface area to be designed, not just the happy path. Often, designers don't see the whole picture up front because they're working from a theoretical model of how things might work, not a model that has been battle-tested in the real world. By building an accurate map of the world, your designers become better equipped to design a clear and delightful interface.

Convincing your boss or the upper-level management

The business cares about generating revenue. This goal is often in opposition to spending time on quality software that works well in a variety of adverse conditions. Correspondingly, these are some of the hardest people to convince. You must show that changing the way you do design and development will positively impact the bottom line.

People have a very low tolerance for apps that don't function. Only 16% of those who fail to use an app the first time will return (http://techcrunch.com/2013/03/12/users-have-low-tolerance-for-buggy-apps-only-16-will-try-a-failing-app-more-than-twice/). If one of the 4 billion people that are without a regular Internet connection tries to use your app and can't because they're offline, how long will they stay with yours before uninstalling and switching to a competitor's app?

On the other hand, if you're the only app among your competitors to provide an exceptional offline experience, you're more likely to gain market share and positive reviews from the billions of people impacted by poor offline experiences.

Revenue aside, if there are people in the management chain who care about design, your job is easier. The higher they are in the chain, the easier still. You can argue with them about design on even terms as they're already convinced of its value. If the CEO wants well-designed products, your job as a design emissary is done.

 

Summary


In this chapter, we established a case for why you want to approach application development with an offline-first mindset. In the next chapter, you will build a basic offline to-do app. Each subsequent chapter will build on this base. By the end of the book, you will have transformed this base into a fully capable online app that translates seamlessly between the online and offline worlds.

Open your development environment and let's get started.

About the Author

  • Daniel Sauble

    Daniel Sauble is part UX designer, part developer, and part researcher. He loves enterprise software start-ups and has worked at companies, including Puppet Labs and Sonatype, on problems encompassing configuration management, repository management, and patch management. Ironically, his first foray into book authorship has nothing to do with any of these.

    In his off time, he runs, speaks, writes, and spends time with his family. He has learned that there is nothing more painful than the end of an ultramarathon, more nerve-wracking than your first conference talk, or more satisfying than a long writing project. One day, he may be foolish enough to start his own company, but for now is content to hone his product design skills in the midst of start-up culture.

    Home is the verdant landscape of the Pacific Northwest, but Daniel enjoys a bit of travel now and then. Between travel, family, work projects, and the personality of an INTJ, he doesn’t have much of a social life. He has no illusions that writing a book will change this much. That said, it’s an excellent conversation starter, should the need arise.

    Browse publications by this author

Latest Reviews

(1 reviews total)
Good
Book Title
Unlock this full book FREE 10 day trial
Start Free Trial