Building Document Based Apps

MIKE HESS: Thank you.

Good afternoon! I'm Mike Hess and I'm a software engineer on the iOS Apps and Frameworks team and I'm here with Johannes Fortmann to talk to you about how to make your document-based apps top notch.

Now we have written some sample code for you today which is going to show you how to build the two main components of a document-based app. First, we're going to show you how to build a great Document Browser which will let your users quickly find the documents that they're interested in working on.

Second, we're going to show you how to build a great document editor that interacts properly with file coordination to deal with concurrent readers and writers such as the iCloud drive daemon. We're going to get into this a little bit later.

First, let's focus on the Document Browser.

So what is a document-based app anyways? Well, we think of a document as a single, standalone entity and it is understood to the user as a single entity. A document-based app is just going to be an app which manages a list of these documents and presents them to the users so that they can view them or edit them or rename them.

Keynote, for example, manages a list of Keynote presentation documents. Numbers manages a list of Numbers spreadsheet documents; even Garage Band manages a list of Garage Band song documents. So we would consider all of these to be document-based apps. Now let's get into how we're going to build our Document Browser.

There are four main components of a great Document Browser.

First, we want to list our documents in a way that's meaningful to our users such as here in our sample code we sort our documents by file name. Our user understands the flow of our app.

Second, we think you should use thumbnails for greater document visibility so that just at a glance your users can quickly identify the document that they're interested in working on.

Third, we want to display all documents that are available to our app, including documents that exist in other apps' containers such as this document in the sample which exists in the shared iCloud Drive container.

And fourth, we think it is a good idea to store a recently accessed list of your documents so that users can quickly get back to the documents that they're currently working on. Now let's go into how we're going to discover our documents for our Document Browser. Now, a naive approach might be to use NSFileManager to try to list your documents in the cloud but these results are incomplete.

For example, in iCloud there is a notion of document promises where there's a document that exists there but content has not been made available locally, it has not been downloaded to disk yet, and NSFileManager does not pick up these documents properly. In addition, if you're trying to list your documents using NSFileManager, external documents are not included so you're not listing all of the documents available to your app. Let's take a quick look at this.

Let's say you're using the NSFileManager APIs.

If you're using NSFileManager, you'll properly pick up document one and document two here, which are completely downloaded to disk in our app's container. But you're missing the result of document 3 which is a document promise from iCloud, and you're also missing document 4 which exists in another app's container but our user has granted our app access to that document. So you don't really want to use the NSFileManager APIs when you're listing your documents.

Instead, you want to use NSMetadataQuery.

Let's take a look at how NSMetadataQuery works.

NSMetadataQuery will pick up all the documents that are available to your app, including document 3 which is the document promise, and document 4 which exists in another app's container but the user granted our app permission to view that document. Now it is important to note here that document 5 which is in another app's container but the user has not granted our app permission to view is still not included in the NSMetadataQuery results because that would be a privacy leak if we showed that document to the user.

So let's use NSMetadataQuery for discovering our documents in the cloud.

Now how does this flow work from your app? Well, first of all you're going to create your NSMetadataQuery.

Then NSMetadataQuery is going to go through an initial gather phase where it lists all of the documents that are currently available to your app. Once this initial gather phase has completed, you will get a notification and then you just have to display these initial documents on your main queue in your app's UI.

But NSMetadataQuery doesn't stop there. In addition, you will receive update notifications as state in the cloud changes such as here the iCloud Drive daemon downloaded a new document to our app's container and we were notified in our NSMetadataQuery about this document.

Then you just need to compute the animations from the changes, such as here we may want to insert a CollectionView cell into our CollectionView, and then just display this updated UI on our main thread.

Now that we know how to discover our documents, let's get into how we're going to make our UI better with document thumbnails.

Now, we think it is a good idea to display thumbnails in your UI because it gives visual context to your user. That way your user can just at a glance identify the document that they're interested in working on because they have a great thumbnail so they can quickly identify it. Now new in iOS 9, thumbnails are actually generated for you automatically for certain document types that are well-known, such as large image files, for example. Now let's get into the workflow of how you might want to load your thumbnail for display in your app's UI. It is important to note here that loading thumbnails involves loading a potentially large amount of data into memory, which can be slow, so you don't want to block your main queue while loading your thumbnail data. Let's take an example workflow, which would work, which is how our sample code app does it.

So first in our sample code we have a CollectionView and the CollectionView asks us to load a CollectionView cell.

We're going to go ahead and schedule a fetch thumbnail job on a background queue because we don't have the thumbnail cached yet.

Now, we're not going to wait for this fetch thumbnail job to complete. We're going to immediately return a CollectionView cell with a placeholder image so that the user knows that there is something there. At some point in the future, the fetch thumbnail job is going to finish and then we'll just notify our CollectionView that it needs to reload that cell and then we'll just display the cell in our UI with our thumbnail.

All right. Now that we know how to find our documents and display them with great thumbnails, let's get into how to manage a recents list.

Now we think you want to use a recents list because recently accessed documents are often the documents that a user is currently working on, so it is a good idea to store a list of these documents that your user can quickly get back to them. Now, a naïve approach, again, might be to use NSURLs to store a recents list of the recently accessed documents, but this suffers from many similar pitfalls as NSFileManager did earlier.

Let's take a quick look at this.

So let's say we store a list of NSURLs to our documents which are the recently accessed documents. But then the iCloud Drive daemon moves the document while our app isn't running, such as here, it moved it into a new folder.

The NSURL is now a broken reference and will not resolve to the updated location of our document on disk so we can't really rely on this to store our recents list.

The correct way to store a recents list is with security scoped bookmarks.

Here, if we store a security scoped bookmark to this document and again the iCloud Drive daemon moves this document into a folder, the bookmark will update automatically to resolve to the document's new location on disk so we want to use security scope bookmarks when managing our recents list. And with that, I would like to get into a quick demo for how we're going to manage our recents list and how to load thumbnails.

All right. Let's go ahead and launch up our sample code here. And we haven't loaded thumbnails yet into our app.

But for example, if I open the iCloud Drive app using the new Multitasking feature, we can tell that the thumbnails for these documents are actually there, so we just need to load them to display in our app's UI. Let's go ahead and look at this in code. All right. So first of all, let's talk about how we're going to manage our recents list in code.

The important thing here is when we're saving our object here we are bookmarking this document using the 'bookmark data with options' method and it is important here to pass the 'suitable for bookmark file' option so that we can resolve it properly later.

Then on our app launch we just have to call the NSURL Constructor method of 'by resolving bookmark data' with the bookmark that we have saved previously, and we'll get a URL, which is the updated location of our document on disk.

Now, it is important here that with this returned URL we need to call 'start accessing security scope for resource, in case this document is a document in another app's container, or we won't be able to read this document because -- read properties from this document because this will extend our Sandbox to have access to this document.

Now for thumbnails we've written this great thumbnail cache class for you in the sample code, which is going to cache our thumbnails for our app.

It takes care of a lot of the heavy lifting for us, such as scheduling, thumbnail loading on background queues, et cetera, and notifying our CollectionView that we need to reload cells. Now the only thing we have not implemented yet is just this block of code right here, which will load our thumbnail from disk. All we have to do is call the NSURL method of 'get promised item resource value for key' on the URL of the document with a thumbnail dictionary key, and we'll get a dictionary of thumbnails. Then we just need to extract the UIImage from the dictionary and return it to our thumbnail class so that we can display it. Now it is important here to use the 'get promised item resource value for key' instead of the 'get resource value for key' method because the document may not have its content available locally yet, so we can display our thumbnail even if it is not downloaded yet.

Then all we have to do is redeploy and we have some great thumbnails in our app, which load in the background so they don't block our scrolling as we scroll through our sample code.

All right. Let's get back to the slides. So what have we learned about building a Document Browser? First, we learned that we want to discover our documents using NSMetadataQuery as opposed to other methods so that we can discover all the documents that are available to our app.

Second, we have learned that we want to display thumbnails in our app's UI so that we can build some great UIs and our users can just quickly identify the documents they're working on.

And finally, we have learned we want to store our recents list using bookmarks as opposed to other methods so that users can quickly get back to the documents that they're currently working on. And with that, I would like to welcome Johannes Fortmann to the stage to talk to you about building the document editor.

JOHANNES FORTMANN: Thank you, Mike. Now, Mike has shown you how to build a beautiful Document Browser in your application. And of course that's something that's very nice for our app but equally as important or possibly even more important is the part of your application where your user can go and load and edit documents. After all, that's why they're trying to use your application.

Now, before we go into the whole loading and writing out change documents, we have to take a quick detour into a concept called file coordination.

Now, what am I talking about here? Well, in our new modern multitasking-based world we have this concept of multiple apps being able to access and display the same file. As an example we could have the iCloud Drive app displaying an overview of all the files that are in your document container at the same time as your app is running and the user is actually editing this document.

Or as a more conservative approach, even if your user is not actually using this two-up view of Multitasking, there is always going to be the case that the iCloud Drive syncing daemon may want to access the document to sync it up to the cloud while your user is editing. In fact, that is a really, really common case, because the user is in the middle of editing the document, they're saving this document to disk, and of course now it is changed, so the iCloud Drive daemon wants to make sure that it is up-to-date in the cloud.

So that's a really common case. Let's have a look at this specific case where your user is in the middle of editing the document on disk. And the way this looks is that your app is, of course, running. And the user is making some edits and in the meantime your application is going through auto-saving, do periodical writes of this document to disk. So we're going to have a write at some point, and at a later time we're going to have like the user's editing, and changing the document, we're going to have another write.

Cool. Now let's just assume for a moment that our user is taking full advantage of the multitasking feature and is at the same time as they're editing this document, also launching another application, and this other application might have a previous reference to this document, and will now immediately doing state restoration for example, try to read this document from disk.

Now as you can see here, this is a bad situation because we're reading this document at the same time as the other application is writing it. Well, that's our application actually. So we're going to get this inconsistent read which, of course, is very unfortunate. We're in the middle of writing this document at the same time the other application is reading it. The data is halfway written to disk. The other half is not. And the other app may not know what to do with this weird, inconsistent data. That's a bad situation. And likewise, even if we somehow manage to live around this, after our second write, remember, we're still in the first application editing this document. The other application will now still be displaying your document and this document is now being displayed in an old version in the other application, so we have got this issue of having a stale display. And that is, of course, very unfortunate. Now conveniently we have two solutions for you here. And that is, first of all, we have this concept of file coordination. File coordination is a distributed reader/writer lock mechanism. And what that means is that, while there can be at any time multiple readers for a document, there can only ever be one writer and the one writer excludes all other readers from reading.

That means that if both our applications are using proper file coordination as they will if they're using UIDocument, which implements these mechanisms, then our read will be moved to a time after our write has finished and in such a way we have always a consistent picture of this document. That's very nice.

Now, there is another mechanism I promised you too. And the other mechanism here is NSFilePresentation. NSFilePresentation is a distributed modification mechanism. What that means is that our file coordination will automatically tell every other file presenter that's been registered for your document that it has been written to disk and that this file presenter has to go and update itself.

That way we immediately get a notification after our second write and we can be sure to update our UI.

Cool. So that's how we can be able to make sure that we have always a consistent picture of our documents. But what documents? Of course, we first need to create some documents to be able to actually display and have the user edit them. So let's have a look at that.

What are our goals in creating these documents? Well, let's imagine as our sample app implements, we have this little plus button in the top right corner, and this plus button, well, the user taps it, we will maybe show a template dialogue, something, but in any case at some point we'll create a new document. And our main goal here is to give the user a consistent display that's up-to-date at all times. So it wouldn't help a lot if the user tapped this little plus button and now we wouldn't get an update in our Document Browser and the user is confused and doesn't know what happened and will possibly tap the plus button a few more times and now we have five different new documents. Not good. Now that's the situation that we might end up in without using file coordination because, of course, what this means is that we create a document on disk and at some point later the iCloud Drive daemon notices that there is a new document here and informs our app. But this delay can be half a second or something, possibly even more if the daemon is busy at this time. And this exact delay is what we want to avoid. Now, conveniently, if you're using a coordinated operation, this is done directly for you.

The coordinated operation works in conjunction with the NSMetadataQuery that you're using to display documents in your browser, and basically loops around after the coordination has finished and immediately tells your running query that there's a new updated document.

That way we get rid of this ugly delay.

Of course, there's another slight caveat here, which is that since we're writing to disk, we're writing anything to disk, it can take a bit of time. And of course doing any operation that can take a bit of time on the main queue is unfortunate because it can block the main queue and thus look like a stutter in your application to the user.

Now, the solution to this is, of course, easy. We want to use a background queue to dispatch this coordinated operation and make sure that our operation is not blocking the main thread. Now conveniently we're still -- since we're using this coordinated operation -- we're still getting the immediately updated display in our UI because our NSMetadataQuery is still updated directly.

You don't even have to bounce this information back to the main queue because we're updating the NSMetadataQuery directly. Cool. Now, another common operation is deleting a file.

That's a totally reasonable thing for your user to do. They're done with this document. They want to get rid of it so it doesn't clutter up their workspace. And the basic idea behind deletion is exactly the same. We'll coordinate a write on our background queue, perform the deletion during this write, and immediately loop back to the main queue through the NSMetadataQuery to update in time.

Cool. So that's how you manage creating and deleting documents on your background queue. So let's have a look at what you actually want to do with these documents. And that, of course, is you want to display them, meaning you want to read and write them. Now, we strongly, strongly, strongly suggest that you use UIDocument for reading and writing documents. UIDocument implements both the NSFileCoordination calls to make sure that at any time you are reading and writing documents in a coordinated manner, as well as implementing NSFilePresentation to make sure that you can be immediately informed that this document has changed and can update it in your display.

So let's have a look at how to read a document. You create a UIDocument object and simply call the 'open with completion' method on this UIDocument instance. And what this method does is it will take out a coordinated read on a background worker queue.

That way your application stays perfectly performant and responsive, while at the same time your document is going to read itself in. Now all that is there that's left to do for you, is that you implement the 'load from contents' method. And this method will simply get called within the coordinated read so it is encapsulated by this coordinated read, meaning it is totally safe to read anything you want in there, from the document, mind you. And all you have to do is basically take the contents and fill in your document data from them. Now, there's another method here that you can alternatively choose to implement, which instead of taking a blob of data, it takes an NSURL and you can use that method to, for example, stream document, there may be situations where that's more reasonable for you to do, because, for example, the document format on disk is very different than what you want in memory. Now, after this is done, we will simply loop back to the main queue and call the completion block that you provided for us. And in that completion block, you can go and, for example, push your new interface updated for your document.

Cool. So that's how reading a document works. But as you remember, Mike told you about this concept called promises. And a promise is basically a file that the iCloud Drive daemon promises to your app is actually there, but it is not yet downloaded.

And what that means is that a read on this document may trigger a download.

Now, of course if you've lived in the real world as we all have, obviously, downloads can be terribly slow at times, and depending how your document size looks, this may take a while. And your user may perceive this download as a failing of your app, which is totally unfair, it is not your fault that this download is being done over a slow network connection and takes a while. And so, for you, new in iOS 9, we expose a way to display progress on UIDocuments using the new NSProgressReporting protocol.

Now, implementing this is very simple. The NSProgressReporting protocol exposes a progress property on your UIDocument instance, and this progress property is filled in by us to display to you what kind of progress we have. So it is basically a simple percentage of the download state.

Now, we expose this NSProgress property through a state change, so the way that you display this is that you listen for a state change notification on your document state, and when the state changes you look at the new flag that says, 'hey, I have a progress that I'm exposing here.' And then you simply display this progress. Now, displaying a progress on, for example, a UIProgress view, used to be a little bit complicated because it exposes a property that you have to key value observe to put it into this progress view. And also we realized that, of course, and also new in iOS 9, we exposed an observe progress property on the UIProgress view that will enable you to just plump the NSProgress directly into the progress view. You simply assign the NSProgress to the progress view's 'observe progress' property and it will automatically update its count .

That's very convenient for you.

Thanks. All that's left for you to do is to listen for the next state change which signals, 'hey, we're done with this progress.' And at that point you want to probably undisplay your UIProgress view and get ready for displaying the new document controller.

Cool. So that's reading documents. Of course, we also want to write documents. And writing documents is very symmetric to reading documents. The way we write documents is that we also take out a coordinated file coordination on a background queue. Now, this one is slightly different in that you're not starting it but rather UIDocument will automatically notice that now is a convenient time to save the document. For example, because for a while there have not been any edits incoming, or it has been a time since the document was last saved, or the user is putting your application into the background so now would be a really convenient time to save the document. But basically the way this is done is very symmetrical to reading a document, we simply call the 'contents for type' method on your UIDocument instance, and you fill in the NSData instance that you then return from there.

Very nice. There is one additional thing here, and that is that this is a convenient time to write a thumbnail. As Mike told you, for some very specific document formats we'll automatically generate thumbnails, but chances are that you're not building the exact thing that we're building, and thus if you're not building, using any of our very, very well-known formats like images or simple text, then you will want to out write thumbnails on your own.

Now, the way you do this is that we will call the 'file attributes to write to your own' method on your UIDocument. And in that method you simply return a UIImage instance as part of your attribute dictionary. And this instance we'll write out contained in the same coordinated write that's writing your document. That means that if the user has at this time the Document Browser up in a separate pane, they'll not see an inconsistent state. Cool. The important thing here is to keep in mind that this is being called on a background queue. And what that means is that you cannot under any circumstances use UIViews to write -- to render your thumbnail. UIViews are not thread safe so you have to make sure that your thumbnail rendering is being done using, for example, Core Graphics, or textKit, or any of the other thread safe rendering mechanisms that we provide on our platform. In iOS 8, the only mechanism to access a document was through a copy. And the reason for that is that applications in general do not have access to each other's Sandbox. So if we have two application Sandboxes, the only way to move a document from a Sandbox to another Sandbox was that the first application caused a copy to be made in the other application Sandbox. Now, we relaxed this thing a little bit through the use of the 'UIDocument Menu View Controller, which allowed your application to do a pull of a document in another application's container. But in general you could not simply open a document. And every open of this document would cause a copy.

Now, of course, doing all of these copies is very inconvenient, because now you have a copy of your document in the other application's container and the user is going to modify it so you have a second version of this document. And now, for example, the user is going to want to open this document back in your application and that causes a third copy to be made. Now you have these three different versions of documents floating around. And that's very unfortunate, because the user gets confused and doesn't know which version is the most recent.

And it is just not a great situation.

Now in iOS 9, we have this nice new mechanism called 'Open in place.' And what this means is that your application, through use of the document interaction controller, can grant another application access to a document. And this document is the exact same document, this is not a copy but rather a reference to this document.

What that means is, of course, that the other application is able to simply make edits that are then, through the magic of file presentation, are directly reflected in your application, which, of course, is very nice for the user. And this includes files that are open from the iCloud Drive app and from Spotlight.

That means that any time your user browses their documents in the iCloud Drive app, we will directly be able to open that document in place.

The mechanism behind this is very similar to the mechanism used by the 'Document Menu View Controller.' That means that if your application is already supporting that, it is super easy for you to also support this mechanism. And even if you're not currently supporting the UIMenuViewController, it is super easy for you to adopt this, because there is really no big magic here. You get a URL and put it into a UIDocument instance that's you then display. Let's have a quick look at how you want to support this. As I said, it is super simple. First of all, you have to tell us that you support it. Your app, remember, is not actually, possibly it may not be launched at the time we're trying to figure out whether we want to open this file in place. So you have to tell us beforehand. And you do that by adopting the 'LS Supports Opening Documents In Place' key in your Info.plist. You simply set that to yes or true, depending on whether you are Objective-C or Swift. And that tells us that your application is able to handle this. Now there is also a bit of code you'll have to write, and that comes in the form of a new delegate method.

How does that look? Well, let's say you already are opening documents here. And it is a reasonable assumption because, well, you're a document-based app. So you must be implementing a method very similar to the one that we see here. Your method currently must be getting an NSURL, and since that NSURL is a temporary copy that the system made for your app, you have to be able to copy this into your own container and then open this copied file. And this is what this small chunk of code here does. Now, the first thing is to adopt our new method here. And this new method is very similar to the old one. It simply takes an options dictionary that encapsulates the parameters that the previous method has. And, importantly, one of these parameters is the 'open in place' key.

Now, all you have to do is have a look at whether this 'open in place' key is actually true, and if this key is true, then your application should open this file in place. That means it should not make a copy. Simple, you just stop doing something that you must have been doing before. And then now that you have this URL, all you have to do is go ahead and open it whichever way you were opening it before. And that's how simple it is to support open in place.

Cool. Let's summarize what we have learned today.

We have seen how to make your app very performance responsive and beautiful using NSMetadataQuery to display a set of documents and to update these documents in time when changes come in from the cloud. We have seen how to use bookmarks to implement a recents mechanism that allows your users to quickly go back to the state that they were before -- that they were in before. And of course, something we didn't talk about, but which is entirely reasonable, you can also use bookmarks to implement state restoration, which allows your users to directly go back to the previous state that they were in. We have seen how to use thumbnails to make your application beautiful, and how to implement progress display to make sure that your user is always aware of what happens in their applications. And last but not least, we have seen how easy it is to implement open-in-place. Open-in-place is a great new mechanism that allows your users to directly open documents in your application without having to make multiple copies.

Now all of these concepts are beautifully displayed in a sample code that we published today. And the sample code is basically the application that Mike showed you previously in the demo. It implements all of these features that are creating documents, displaying documents in a Document Browser, animating changes on these documents, and, of course, writing thumbnails, and, of course, open-in-place.

For technical support, we would like to direct you to our forums.

And we also have amazing documentation under the link that you see here. If you're interested in learning more about using the 'UIDocument Menu View Controller' to implement the pull mechanism as opposed to the pull mechanism that open-in-place implements, so that your application can pull documents from another application's container, or if you're generally interested in how to implement UIDocument-based applications, we'd like to point you to our session from last year, that's session 234 last year, and, of course, that's online as well. And with that, thank you for your time and have a great afternoon.

[ Applause ]