Chapter 6. Text and Language

This chapter explores the practical side of implementing text- and language-related AI features in your Swift apps. Taking a top-down approach, we explore five text and language tasks and how to implement them using a variety of AI tools.

Practical AI, Text, and Language

The five text and language tasks that we explore in this chapter are:

Language Identification

Determining what language some text might be in.

Named Entity Recognition

Identifying the elements of text that are people, places, or organizations.

Lemmatization, tagging, tokenization Identifying the lemma of every word in a string, finding the parts of speech (verbs, nouns, and so on), and splitting a string up by words.

Sentiment Analysis

Determining whether some text has a positive or negative sentiment.

Custom Text Classifiers

Another way of classifying text for sentiment, extending Apple’s tools.

Note

In Chapter 8, we also look at generating text. We put that task there because we think it’s more closely related to generating things than it is to text. But really, you’re probably reading the whole book, so it doesn’t matter where it goes.

Images, human movement, and sound might be flashy, but the majority of apps you’ll build in your life will also, or perhaps primarily, deal with text. Humans generate vast amounts of text, and it’s often useful to be able to use a clever machine to figure out what’s going on with text so that you can make decisions or show the user something contextual relating to it. In this chapter, we tackle the problem of text classification. Specifically, we’re going to look at implementing an app that can perform sentiment analysis on some text and determine whether its sentiment is positive or negative.

Note

You might see other sources mix and match the terms “text classification,” “sentiment analysis,” “natural language processing,” “opinion mining,” and many others. The authors of this book are of the opinion that they are quite different things. This chapter explores the specific task of sentiment analysis, which is part of the domain of text classification. In doing so, we use natural language processing (NLP) techniques.

Task: Language Identification

Language identification refers to (surprising no one) figuring out what language a string of text might be in. This is actually a very simple practical artificial intelligence (AI) task.

To cut straight to it, we do this task in a Playground:

  1. Create a new iOS-flavor Playground in Xcode, as shown in Figure 6-1.

Note

We’re using iOS because we’re choosing to use iOS. Everything we’re using for this task is available on macOS, too.

pais 0406
Figure 6-1. Creating a new iOS-flavor Playground in Xcode

  1. Add the following imports:

    import NaturalLanguage
import Foundation
import CoreML

  2. Add the following extension on String:

    extension String {
    func predictLanguage() -> String {
        let locale = Locale(identifier: "es")
        let recognizer = NLLanguageRecognizer()
        recognizer.processString(self)
        let language = recognizer.dominantLanguage
        return locale.localizedString(
            forLanguageCode: language!.rawValue) ?? "unknown"
    }
}

    This means that we can ask a String to predictLanguage(), and we’ll get its language back. We do this by setting the locale to “en_US” (for US English), creating an NLLanguageRecognizer, processing the String being used, and getting the dominant language for that String.

  3. Add a String (or in this case, an array of them) for us to identify the languages for the following sentences:

    let text = ["My hovercraft is full of eels",
            "Mijn hovercraft zit vol palingen",
            "我的氣墊船充滿了鰻魚",
            "Mit luftpudefartøj er fyldt med ål",
            "Το χόβερκραφτ μου είναι γεμάτο χέλια",
            "제 호버크래프트가 장어로 가득해요",
            "Mi aerodeslizador está lleno de anguilas",
            "Mein Luftkissenfahrzeug ist voller Aale"]

  4. Test it by iterating through the Strings in the array and calling predictLanguage() on each:

    for string in text {
    print("\(string) is in \(string.predictLanguage())")
}

    You will see something like the screenshot in Figure 6-2.

pais 0602
Figure 6-2. The language identification for our strings

  1. We could change the locale to be somewhere else, for example “es” for Spain, and we’d get back the Spanish names for the various languages instead, as shown in Figure 6-3.

pais 0603
Figure 6-3. The output with our locale set to Spain

Task: Named Entity Recognition

Almost as simple as recognizing language is recognizing the entities in a string. As with language identification, this task relies on using Apple’s Natural Language framework to do the work for us. The Natural Language framework works on texts by assigning tag schemes.

Again, let’s get straight to it and do our work in a Playground:

  1. Create another new iOS-flavor Playground in Xcode.

  2. Add the following `import`s:

    import NaturalLanguage
import Foundation
import CoreML

  3. Add the following extension on String:

    extension String {
    func printNamedEntities() {
        let tagger = NSLinguisticTagger(
            tagSchemes: [.nameType],
            options: 0)

        tagger.string = self

        let range = NSRange(location: 0, length: self.utf16.count)

        let options: NSLinguisticTagger.Options = [
            .omitPunctuation, .omitWhitespace, .joinNames
        ]
        let tags: [NSLinguisticTag] = [
            .personalName, .placeName, .organizationName
        ]

        tagger.enumerateTags(in: range,
            unit: .word,
            scheme: .nameType,
            options: options) {
                tag, tokenRange, stop in

                if let tag = tag, tags.contains(tag) {
                    let name = (self as NSString)
                        .substring(with: tokenRange)

                    print("\(name) is a \(tag.rawValue)")
                }
        }
    }
}

    This means that we can ask a String to printNamedEntities(), and we’ll see it print its name entities.

  4. Add a String on which we can perform named entity recognition:

    let sentence = "Marina, Jon, and Tim write books for O'Reilly Media " +
    "and live in Tasmania, Australia."

  5. Test it by calling printNamedEntities() on it:

    sentence.printNamedEntities()

    You will see something like the screenshot in Figure 6-4.

pais 0604
Figure 6-4. The language identification for our strings

Task: Lemmatization, Tagging, and Tokenization

Lemmatization is one of those things that you’ve probably heard of but aren’t quite sure what it is. But it’s useful for all manner of things.

Lemmatization is a linguistics term that refers to the process of grouping all the forms of a single word so that they can be identified as a single thing. The single thing that identifies them is the lemma.

For example, take the term (a verb) “to walk.” “To walk” can appear as “walk,” “walked,” “walks,” “walking.” To look up any of those in a dictionary, you’d look up “walk.” Not every word has an obvious lemma; for example, the lemma of “better” is “good.”

Lemmatization is useful for things like search tools in your apps: if a user searches for “good,” you probably want to identify things that are also marked “better,” or if your app, for example, deals with photos, and you’ve performed machine-learning classification to establish what’s in each photo, you’d want the search term “mouse” to also present results for “mice,” and vice versa.

Again, let’s dispense with the usual structure and get straight to it:

  1. Create a new iOS-flavor Playground in Xcode.

  2. Add the following imports:

    import NaturalLanguage
import Foundation
import CoreML

  3. Add a sentence on which we can perform lemmatization:

    let speech = """
Space, the final frontier. These are the voyages of the
Starship Enterprise. Its continuing mission to explore strange new worlds,
to seek out new life and new civilization, to boldly go where no one has
gone before!
"""

    In this case, we’ve used the opening monologue to Star Trek. The Jean-Luc Picard version, naturally.

  4. Add an extension on String:

    extension String {
    func printLemmas() {
        let tagger = NSLinguisticTagger(tagSchemes:[.lemma], options: 0)
        let options: NSLinguisticTagger.Options = [
            .omitPunctuation, .omitWhitespace, .joinNames
        ]

        tagger.string = self
        let range = NSRange(location: 0, length: self.utf16.count)

        tagger.enumerateTags(
            in: range,
            unit: .word,
            scheme: .lemma,
            options: options) {
                tag, tokenRange, stop in

                if let lemma = tag?.rawValue {
                    print(lemma)
                }
        }
    }
}

    With this extension, we can ask a String to printLemmas() and get a console output showing the lemmas of the String. Within the printLemmas() function, we create an NSLinguisticTagger, set its scheme to .lemma, and then run it on the String (which is self in this context, because it’s an extension on String).

  5. To test our extension and printLemmas() function, we can call it on the String speech. See Figure 6-5 for the result:

    speech.printLemmas()
pais 0605
Figure 6-5. Lemmatization running on our speech
Note

If you start researching lemmatization—which you should because it’s interesting—you might see it referred to as “stemming.” They’re basically the same thing, as far as their usefulness matters, but in reality, stemming actually just involves stripping plurals and “ings” from words, and lemmatization involves understanding the language in question, and how the vocabulary works.

Tokenizing a Sentence

But what if we just want to split the sentence up by words and don’t really care what the lemma of each word is or what part of speech it is? We can do that too:

  1. Add another function, printWords() to the String extension:

    func printWords() {

    let tagger = NSLinguisticTagger(
        tagSchemes:[.tokenType], options: 0)

    let options: NSLinguisticTagger.Options = [
        .omitPunctuation, .omitWhitespace, .joinNames
    ]

    tagger.string = self
    let range = NSRange(location: 0, length: self.utf16.count)

    tagger.enumerateTags(
        in: range,
        unit: .word,
        scheme: .tokenType,
        options: options) {
            tag, tokenRange, stop in

            let word = (self as NSString).substring(with: tokenRange)
            print(word)
    }
}

  2. Run it on the monologue:

    speech.printWords()

    You’ll see something like the screenshot in Figure 6-7.

pais 0606
Figure 6-6. The parts of speech of the Star Trek opening monologue
pais 0607
Figure 6-7. Printing the words in the Star Trek opening monologue
Note

The process of identifying and printing words is called tokenization.

You might be wondering why we can’t just use a regular expression to split up the sentence by punctuation and spaces. The short answer is that this doesn’t guarantee you’ll end up with every word, and many languages don’t behave the same way that English does in this respect. It’s better to rely on the Apple framework’s understanding the semantics of the language you want to work with wherever possible.

Note

In Chapter 8, as part of a Sentence Generation task (“Task: Sentence Generation”), we manually perform tokenization using regular expressions. We did this to highlight the differences.

Task: Sentiment Analysis

Sometimes, it’s really useful to be able to determine whether something your users said is positive or negative, or generally to be able to derive some kind of organized data from unstructured, unorganized information. IBM estimates that 80% of the world’s data is unstructured (and you’d expect IBM to know what it’s talking about—it has a company song!).

Humans generate vast quantities of unstructured, unorganized text, and our apps and products often need to know what the text is about, what the text means, or the general flavor of the text in order to do something useful with it or provide useful options to the user.

Put simply, performing text classification in order to derive sentiment—sentiment analysis—is a way to bring order to the chaos of human-generated text.

For this task, we look at how we might build a model that allows us to determine the sentiment of some text. This isn’t something that Apple’s provided frameworks can do out of the box, so we’ll actually need to train our own model to do it and then build an app around it.

Building the App

We’re going to start simple here. We need an app that can detect whether what the user has typed in is positive or negative. We all live high-pressure lives, and often act in the spur of the moment. Having an app that lets us check whether that tweet we’re about to send is positive enough might be a good idea. (“Computer! Send Tweet!”)

The app we’re going to build will look something like Figure 6-8 when we’re done.

Note

This book is here to teach you the practical side of using AI and machine-learning features with Swift and on Apple’s platforms. Because of this, we don’t explain the fine details of how to build apps; we assume that you mostly know that (although if you don’t, we think you’ll be able to follow along just fine if you pay attention). If you want to learn Swift, we recommend picking up Learning Swift (also by us!) from the lovely folks at O’Reilly.

The starting point iOS app that we’re going to build, which will ultimately house our sentiment analysis system, has the following components (see Figure 6-9):

  • A UITextView, for a user to type text that will be analyzed for sentiment

  • A UIButton for the user to press when they want to type text in the aforementioned field to be analyzed for sentiment

  • A UIView that will be set to a color that dictates the sentiment we’ve detected in the text (e.g., red or green, for negative and positive, respectively), two UILabels to display a relevant emoji, and a string describing the sentiment

pais 0609
Figure 6-8. Our sentiment classification app, in all its finished glory
pais 0610
Figure 6-9. The starting point for the sentiment analysing app
Tip

If you don’t want to manually build the starting point iOS app, you can download the code from our website and find the project named NLPDemo-Starter. After you have that, skim through the rest of this section and then meet us at “AI Toolkit and Dataset”.

To make the starting point yourself, you’ll need to do the following:

  1. Create an iOS app project in Xcode, choosing the “Single View App” template. Do not select any of the checkboxes below the Language drop-down (which are, as per usual, set to “Swift”).

  2. After the project is created, open the Main.storyboard file and create a user interface with the following components:

    • A UIButton with its title text set to “Analyse Sentiment”.

    • A large, editable, scrollable UITextView.

    • A generic UIView (which will be used to show a color), with two UILabel views within it: one with its title set to “None” or similar, and the other with a neutral emoji, such as ߘஊYou can see an example of our storyboard for the app in Figure 6-10.

      pais 0611
      Figure 6-10. Our storyboard for the sentiment classifier

      After you have the necessary elements laid out, make sure that you add the proper constraints.

  3. Connect the outlets for the user interface (UI) objects, as follows:

    @IBOutlet weak var emojiView: UILabel!
@IBOutlet weak var labelView: UILabel!
@IBOutlet weak var colorView: UIView!
@IBOutlet weak var textView: UITextView!

  4. Connect an action for the UIButton, as follows:

    @IBAction func analyseSentimentButtonPressed(_ sender: Any) {
    performSentimentAnalysis()
}

  5. Declare an attribute for some placeholder text to go in the UITextView:

    private let placeholderText = "Type something here..."

  6. Modify the viewDidLoad() function, making it look as follows:

    override func viewDidLoad() {
    textView.text = placeholderText
    textView.textColor = UIColor.lightGray
    textView.delegate = self

    super.viewDidLoad()
}

  7. Add the following function, which is used to actually ask for a sentiment analysis later, after we add the model:

    private func performSentimentAnalysis() {

    emojiView.text = sentimentClass.icon
    labelView.text = sentimentClass.description
    colorView.backgroundColor = sentimentClass.color
}

  8. Add an extension to the end of the ViewController.swift file, as follows (it’s a fairly large block of code, as per our previous examples, but as usual we’ll explain it in a moment):

    extension ViewController: UITextViewDelegate {
    func textViewDidBeginEditing(_ textView: UITextView) {
        if textView.textColor == UIColor.lightGray {
            textView.text = nil
            textView.textColor = UIColor.black
        }
    }

    func textViewDidEndEditing(_ textView: UITextView) {
        if textView.text.isEmpty {
            textView.text = placeholderText
            textView.textColor = UIColor.lightGray
        }
    }
}

    This extension makes ViewController conform to UITextViewDelegate, which lets us manage the beginning and ending of someone editing a UITextView. We implement two functions that map to that, and change the color of the text when each happens.

  9. Add a new Swift file to the project named Sentiment.swift, and then place the following code in it:

    import UIKit

extension String {
    func predictSentiment() -> Sentiment {
        return [Sentiment.positive, Sentiment.negative].randomElement()!
    }
}

    This code adds an extension on the String class (which comes with Swift) to add a function named predictSentiment(), so we can just ask any object of type String for its sentiment by calling that function.

    At the moment, we just return a random choice between the negative or positive sentiment.

  10. Add an enum below this, in the same Sentiment.swift file:

    enum Sentiment: String, CustomStringConvertible {
    case positive = "Positive"
    case negative = "Negative"

    var description: String { return self.rawValue }

    var icon: String {
        switch self {
            case .positive: return "ߘ䢊            case .negative: return "ߘ⢊        }
    }

    var color: UIColor {
        switch self {
            case .positive: return UIColor.systemGreen
            case .negative: return UIColor.systemRed
        }
    }
}

    This enum creates a new type called Sentiment that has two cases: Positive and Negative. For each case, we define an icon (which returns an emoji) and a color (which returns a color).

  11. Add a launch screen and an icon, if you’d like to (as usual, our starter project has some you can use), and then launch the app in the simulator. You should see something that looks like the image we showed you earlier, in Figure 6-9.

You can type some text into the text field and then tap the button. The color view, emoji, and text label will update with either positive or negative sentiment. Remember that for the moment this is random (because of the code in our extension on the String class, in Sentiment.swift).

Let’s get moving, and add some AI.

AI Toolkit and Dataset

As usual with our practical AI tasks, we need to assemble a toolkit with which to tackle the problem. The primary tools that we use in this case are CreateML, CoreML, and Xcode’s Playgrounds feature.

As we’ve done before, we use Apple’s task-based tool, CreateML, to build a model for our sentiment analysis. Instead of using the CreateML application, we’ll be using CreateML from an Xcode Playground, using it as a Swift framework. It’s a less visual but more flexible approach to building models.

As with the previous practical AI tasks, we use CoreML to implement the model within our Swift application.

To make an app that can determine whether text is positive or negative, we need a dataset with lots of both sentiments. For that, we turn to internet product reviews. Internet product reviews are a great place to find people being very, very negative, and very, very positive about all manner of things. For this dataset, we turn to the boffins at Carnegie Mellon University, who have done the yeoman’s work of acquiring 691 posts that are positive about a certain brand of automobile, and 691 posts that are negative about a certain brand of automobile. You can see some examples of this data in Figure 6-11.

pais 0612
Figure 6-11. A snapshot of the sentiment data that we use in this task

Head over to Carnegie Mellon’s website and download the epinions3.zip file. Unzip the file and then put the output (a file, quite creatively named epinions3.csv) in a safe place.

If you open this file, you’ll see that it’s just a comma-separated list of classes (Neg or Pos) and text (the text of the review of an automobile), as shown in the snapshot in Figure 6-12.

pais 0613
Figure 6-12. Example of the sentiment data
Tip

You’re not restricted to two categories; you could classify as many categories of text as you want. We’re just starting here with two because it makes the app a little simpler to show.

Creating a Model

Now that we have a useful dataset ready to create a model, we turn to Apple’s CreateML to perform the training. In Chapter 7 we use Apple’s Python library, TuriCreate, to train our model, and in Chapters 4 and 5 we use Apple’s CreateML application. But for this task, we use Apple’s CreateML framework directly from a Playground in Xcode.

Tip

We recommend saving the training Playground alongside the projects you create that will consume the model. This will make it easier to re-create and modify your project in the future or work with a different model.

To do this we need to create a new Xcode Playground:

  1. Fire up Xcode and create a new Playground, as shown in Figure 6-13. The Playground needs to be a macOS Playground, not an iOS Playground, because we’re using the macOS-only framework, CreateML.

    pais 0614
    Figure 6-13. Making a new macOS Playground in Xcode

  2. Add the following code to the Playground:

    import CreateML
import Foundation

// Configure as required
let inputFilepath = "/Users/mars/Desktop/"
let inputFilename = "epinions3"
let outputFilename = "SentimentClassificationModel"

let dataURL = URL(fileURLWithPath: inputFilepath + inputFilename + ".csv")
let data = try MLDataTable(contentsOf: dataURL)
let (trainingData, testingData) = data.randomSplit(by: 0.8, seed: 5)

    This code imports the CreateML and Foundation frameworks, and sets up some working variables:

    • inputFilepath stores the path to the datafile we’ll be using. Update this to point to wherever you’ve saved the dataset we were working with in “AI Toolkit and Dataset”.

    • inputFilename stores the name of the file. If you didn’t modify the downloaded file, this should already be correct.

    • outputFilename sets a name for the model that we’d like CreateML to output.

    • data stores an MLDataTable of the data, based on the inputFilepath and inputFilename. MLDataTable is a type provided by CreateML, which is basically like a spreadsheet for data with which you want to train a model. Each row in an MLDataTable is an entity, and each column is a feature of that entity that your training might be interested in. You can learn more about MLDataTable in Apple’s documentation, but we unpack it a little more as we work on practical AI tasks throughout this book.

    • trainingData and testingData stores a split of the data we’ve read in, taking 80% of it for training, and 20% of it for testing.

  3. In this step, we actually do the training. Add the following code below the variables:

    print("Begin training...")

do {

    // Final training accuracy as percentages

} catch {
    print("Error: \(error)")
}

    It’s within this do-catch is that we’ll actually perform the training.

  4. Inside the do-catch, add the following:

    let sentimentClassifier = try MLTextClassifier(
    trainingData: trainingData,
    textColumn: "text",
    labelColumn: "class")

    This creates an MLTextClassifier, which is a type provided by the CreateML framework.

    The MLTextClassifier allows us to create a text classifier based on associated labels in the input text. You can learn more about the MLTextClassifier in Apple’s documentation, but we also explain it a little more, later on in this chapter.

    In this case, we’re creating an MLTextClassifier called sentimentClassifier, passing in the trainingData (which is 80% of the data we downloaded) in the form of an MLDataTable. We instruct the MLTextClassifier that we want it to look at the column named “text” for the text source, and the column named “class” for the label source. If you look back to the snapshot of the data we showed in Figure 6-12, you’ll notice these column names match the data we imported here.

    This line of code actually creates and trains the model. You could, kind of, stop here. But we’re not going to.

  5. Immediately below this, add the following code:

    let trainingAccuracy =
    (1.0 - sentimentClassifier.trainingMetrics.classificationError)
        * 100

let validationAccuracy =
    (1.0 - sentimentClassifier.validationMetrics.classificationError)
        * 100

print("Training evaluation: \(trainingAccuracy), " +
    "\(validationAccuracy)")

    This defines some accuracy variables for us to store some information about the model we just trained in. Specifically, we store both the training accuracy as well as the validation accuracy as percentages and then print them out.

  6. Add the following code immediately below this:

    // Testing accuracy as a percentage

// let evaluationMetrics =
//    sentimentClassifier.evaluation(on: testingData) // Mojave

let evaluationMetrics = sentimentClassifier.evaluation(
    on: testingData,
    textColumn: "text",
    labelColumn: "class") // Catalina

let evaluationAccuracy =
    (1.0 - evaluationMetrics.classificationError) * 100

print("Testing evaluation: \(evaluationAccuracy)")

let metadata = MLModelMetadata(
    author: "Mars Geldard",
    shortDescription: "Sentiment analysis model",
    version: "1.0")

try sentimentClassifier.write(
    to: URL(
        fileURLWithPath: inputFilepath + outputFilename + ".mlmodel"),
    metadata: metadata)

    This evaluates the model, using the 20% segment of the data we separated earlier, stores, and then prints some evaluation metrics. It also sets the model metadata, such as the author, a short description, and version number, and then writes out an .mlmodel file for use with CoreML.

Note

This model won’t take nearly as long to train as the models in the earlier chapters, because it’s a much simpler operation than image classification, sound classification, and the like. It could take a few minutes, but not much longer. No time for Person of Interest here, sorry.

If you run the Playground, you should end up with some text output about the model in the console, as well as a new SentimentClassificationModel.mlmodel file (if you didn’t change our filenames).

Incorporating the Model in the App

As usual, at this juncture we have both a starting point app and a trained model. It’s time to combine them to make an app that can actually perform sentiment analysis on text that a user has entered.

Tip

You need to either build the starting point yourself, following the instructions in “Building the App”, or download the code from our website finding the project named NLPDemo-Starter. We’ll be progressing from that point in this section. If you don’t want to follow along and manually work with the app’s code to add the sentiment analysis features, you can also work with the project named NLPDemo-Complete.

As usual, we’re going to need to change a few things to get the app working with our model.

Note

We recommend working through the next section even if you download our code. Just read along and compare what we did in the code to the book so that you get an understanding of how it’s working.

First, let’s make some changes to the enum for Sentiment.swifts, Sentiment:

  1. Add an extra case at the beginning of the Sentiment enum, with the extra line covering a lack of sentiment:

    case positive = "Positive"
case negative = "Negative"
case neutral = "None"

  2. Similarly, add a default case to the switch statement in the icon variable, to account for a lack of sentiment:

    var icon: String {
    switch self {
        case .positive: return ""
        case .negative: return ""
        default: return ""
    }
}

  3. For the color, return gray if there’s no sentiment found:

    var color: UIColor {
    switch self {
        case .positive: return UIColor.systemGreen
        case .negative: return UIColor.systemRed
        default: return UIColor.systemGray
    }
}

  4. Add an initializer to the Sentiment enum, where the raw value must precisely match the class labels from the training data (so in this case, “Pos” and “Neg”):

    init(rawValue: String) {
    // initialising RawValues must match class labels in training files
    switch rawValue {
        case "Pos": self = .positive
        case "Neg": self = .negative
        default: self = .neutral
    }
}

    Next, we need to update the predictSentiment() function, near the top of the Sentiment.swift file, to actually make use of a model.

  5. Below the import statement, add the following to bring in Apple’s natural language framework:

    import NaturalLanguage

  6. Change the predictSentiment() function to look like the following:

    func predictSentiment(with nlModel: NLModel) -> Sentiment {
    if self.isEmpty { return .neutral }
    let classString = nlModel.predictedLabel(for: self) ?? ""
    return Sentiment(rawValue: classString)
}

    This new function takes an NLModel as a parameter (we, creatively, call it nlModel), returns a Sentiment (which is our own enum type) and checks whether nlModel is empty (returning Sentiment.neutral if it is). Otherwise, it asks nlModel for a prediction based on its contents (remember the predictSentiment() function is an extension of String) and returns that prediction as Sentiment by initializing a new Sentiment using the initializer we just made.

At this point you can drag the SentimentClassificationModel.mlmodel file into the project’s root, letting Xcode copy it in as needed.

We also need to make some changes to ViewController.swift in order to make this work:

  1. Add a new import below the existing one, to bring in Apple’s language framework (as we did for Sentiment.swift):

    import NaturalLanguage

  2. Add the following new attribute below placeholderText:

    private lazy var model: NLModel? = {
    return try? NLModel(mlModel: SentimentClassificationModel().model)
}()

    This attribute, model, stores a reference to our actual model. If your model is not called SentimentClassificationModel.mlmodel, you’ll need to change this as appropriate here.

  1. Change the following code in the performSentimentAnalysis() function, removing this

    let text = textView.text ?? ""
let sentimentClass = text.predictSentiment()

    and replacing it with this:

    var sentimentClass = Sentiment.neutral

if let text = textView.text, let nlModel = self.model {
    sentimentClass = text.predictSentiment(with: nlModel)
}

    This code creates a new Sentiment (our custom type, from Sentiment.swift), setting it to neutral, and then gets the text from our textView, uses the model attribute we created a moment ago (which is a reference to our model). It then requests a sentiment (using the predictSentiment() function with which we’ve extended String, within Sentiment.swift) and stores the result in the new Sentiment we just created.

The rest of the code, which is unchanged, reads the properties of the Sentiment sentimentClass we just created (and hopefully stored a predicted sentiment in) and updates the relevant UI elements to match the predicted sentiment.

Everything should be ready to go now. Launch the app in the simulator and try it out. Figure 6-14 shows the results from our app.

pais 0609
Figure 6-14. The final sentiment classifier

Task: Custom Text Classifiers

In the previous section, we trained our own custom sentiment classifier and implemented it from scratch in an iOS app.

There’s another way to do something similar. In this task, we make a custom text classifier that works with the text system we’ve been using in the earlier tasks. We use CreateML’s MLTextClassifier to train a model again, as we did in “Task: Sentiment Analysis”, but, here, we show you a different way to use the model.

Instead of using the trained MLTextClassifier model as a more generic CoreML, use it with NLTagger and NLTagScheme, which lets us call our custom model as if it were one of Apple’s provided models (such as those we used earlier for “Task: Language Identification”, “Task: Named Entity Recognition”, and “Task: Lemmatization, Tagging, and Tokenization”).

AI Toolkit and Dataset

As usual with our practical AI tasks, we need to assemble a toolkit with which to tackle the problem. The primary tools that we use in this case are CreateML, CoreML, and Xcode’s Playgrounds. As with the “Task: Sentiment Analysis”, we’re using Apple’s task-based tool, CreateML, via an Xcode Playground, to train a model.

We use the Kaggle restaurant review dataset, which is similar to the one we used earlier in “Task: Sentiment Analysis”.

We’ve converted it to JSON for ease of parsing, as shown in Figure 6-15. You can find the Reviews.json file in the NaturalLanguage-Demos folder, which is available in our resource download on our website.

pais 0616
Figure 6-15. A sample of the review data, in JSON

Creating a model

With our dataset chosen, let’s fire up CreateML in a Playground to do some training. This process is very similar to the process we used earlier in “Creating a Model”:

  1. Create a new macOS Playground in Xcode named TrainCustomTagger. Ours is shown in Figure 6-16.

pais 0617
Figure 6-16. Our Playground for training a tagger

  1. Add the following imports:

    import Foundation
import CreateML

  2. Add some code to load the raw data, create an MLDataTable, and split the data into training and test sets:

    let dataPath = "/Users/parisba/ORM Projects/Practical AI with Swift " +
    "1st Edition/PracticalAIwithSwift1stEd-Code/ChapterXX-" +
    "NaturalLanguage/Reviews.json"

let rawData = URL(fileURLWithPath: dataPath)

let dataset = try MLDataTable(contentsOf: rawData)

let (trainingData, testData) = dataset.randomSplit(by: 0.8, seed: 7)

  3. Create an MLTextClassifier model, and set up the evaluations:

    let model  = try MLTextClassifier(
    trainingData: trainingData,
    textColumn: "text",
    labelColumn: "label")

let metrics = model.evaluation(
    on: testData,
    textColumn: "text",
    labelColumn: "label")

let accuracy = (1 - metrics.classificationError) * 100
let confusion = metrics.confusion

  4. Write out the CoreML model:

    let modelPath = "/Users/parisba/ORM Projects/Practical AI with Swift" +
    "1st Edition/PracticalAIwithSwift1stEd-Code/ChapterXX-" +
    "NaturalLanguage/ReviewMLTextClassifier.mlmodel"

let coreMLModel = URL(fileURLWithPath: modelPath)
try model.write(to: coreMLModel)

Run the Playground. The output should show the training progress, the accuracy from testing, and a confirmation that the file was written out successfully, as shown in Figure 6-17.

Tip

We recommend saving the training Playground alongside the projects you create that will consume the model. This will make it easier to re-create and modify your project in the future, or work with a different model.

pais 0618
Figure 6-17. The custom tagger training

Using the model

We’re not going to step through the creation of a full app for this task, because it’s derivative of our sentiment analyzer from “Task: Sentiment Analysis”.

Tip

We did actually build an app for this, just in case you want to look at it. To see our app for this task, look for CTDemo in the resources available on our website.

To use the custom NLTagger model we’ve trained using MLTextClassifier, create a new Swift file in the project you want to use it in (ours is called ReviewTagger.swift) and then do the following:

  1. import the necessary frameworks:

    import Foundation
import NaturalLanguage
import CoreML

    We’re specifically after NaturalLanguage and CoreML, so we can use CoreML to work with models, and NaturalLanguage to work with language-specific features.

  2. Drag the trained model into the project in question and allow Xcode to copy as necessary.

  3. Create a class to represent your tagger:

    final class ReviewTagger {

}

  4. Add some useful variables:

    private static let shared = ReviewTagger()

private let scheme = NLTagScheme("Review")
private let options: NLTagger.Options = [.omitPunctuation]
Note

Make sure the line where you define the modelFile points to the name of your classifier model. It might be different from ours (ReviewMLTextClassifier).

  1. Create an NLTagger:

    private lazy var tagger: NLTagger? = {
    do {
        let modelFile = Bundle.main.url(
            forResource: "ReviewMLTextClassifier",
            withExtension: "mlmodelc")!

        // make the ML model an NL model
        let model = try NLModel(contentsOf: modelFile)

        // connect model to (custom) scheme name
        let tagger = NLTagger(tagSchemes: [scheme])
        tagger.setModels([model], forTagScheme: scheme)

        print("Success loading model")
        return tagger
    } catch {
        return nil
    }
}()

  2. Stub out the necessary init() function:

    private init() {}

  3. Create a function to call for a prediction:

    static func prediction(for text: String) -> String? {
    guard let tagger = ReviewTagger.shared.tagger else { return nil }
    print("Prediction requested for: \(text)")
    tagger.string = text
    let range = text.startIndex ..< text.endIndex
    tagger.setLanguage(.english, range: range)
    return tagger.tags(in: range,
        unit: .document,
        scheme: ReviewTagger.shared.scheme,
        options: ReviewTagger.shared.options)
    .compactMap { tag, _ -> String? in
        print(tag?.rawValue)
        return tag?.rawValue
    }
    .first
}

  4. Create an extension on String, allowing you to request a prediction using the ReviewTagger class we just made:

    extension String {
    func predictSentiment() -> Sentiment {
        if self.isEmpty { return .neutral }
        let classString = ReviewTagger.prediction(for: self) ?? ""
        return Sentiment(rawValue: classString)
    }
}

    Here, we use the Sentiment enum that we created for “Task: Sentiment Analysis” to return an emoji for the sentiment.

  5. You could also directly use our ReviewTagger:

    let tagger = ReviewTagger()
let testReviews = [
    "I loved this place and it served amazing food",
    "I did not like this food, and my steak was off",
    "The staff were attentive and the view was lovely.",
    "Everything was great and the service was excellent"
]
testReviews.forEach { review in
    guard let prediction = tagger.prediction(for: review) else { return }
    print("\(review) - \(prediction)")
}

Instead of using MLTextClassifier to train a model, you could also use MLWordTagger to train a model using CreateML. MLWordTagger models can be used exactly as we did here (with a custom tag scheme), but they’re designed to be used for recognizing words relevant to your app, like product names or unique points of interest.

For example, using MLWordTagger, you could build an AI-powered system that understood which bits of a String were, for example, alien races in a sci-fi universe that your app (or perhaps game) was dealing with:

  1. If you had a dataset that outlined some example sentences, identifying which bits were aliens, such as this:

    {
    "tokens": ["The", "Vorlons", "existed",
               "long", "before", "humanity!"],
    "labels": ["other", "alien", "other",
               "other", "other", "other"]
},
{
    "tokens": ["The", "Vorlons", "are", "much",
               "older", "than", "the", "Minbari."],
    "labels": ["other", "alien", "other", "other",
               "other", "other", "other", "alien"]
}
Tip

As you might have gathered from our use of it, JSON is an excellent way to work with text in machine learning.

  1. You could then load it into an MLDataTable, as we did earlier, and train an MLWordTagger on it. With the resulting model, you could define a tag scheme:

    var alienScheme = NLTagScheme("Alien")

  2. And the NLTag you want it to look for:

    var alienTag = NLTag("alien")

Then, you could run the MLWordTagger on sentences, and if you had sufficient training data, it would be able to flag which parts of a sentence were alien races, based on the training.

Next Steps

That’s everything for our text and language chapter. We’ve covered some common text- and language-related practical AI tasks that you might want to accomplish with Swift, and we used a variety of tools to do so.

We performed five practical AI tasks:

Language Identification

Determining what language some text might be in using Apple’s Natural Language framework.

Named Entity Recognition

Identifying the elements of text that are people, places, or organizations, again using Apple’s Natural Language framework.

Lemmatization, tagging, tokenization

Identifying the lemma of every word in a string, finding the parts of speech (verbs, nouns, and so on), and splitting a string up by words, still using Apple’s Natural Language framework.

Sentiment Analysis

Figuring out if some text has a positive or negative sentiment.

Custom Text Classifiers:: Building our own text classifier on top of Apple’s Natural Language framework.

In Chapter 11, we look at what happened under the hood, algorithm-wise, for each of the tasks that we explored in this chapter (“Text and Language”).

If you want to take language and text a step further with practical AI, we recommend taking a look at BERT. BERT stands for Bidirectional Encoder Representations from Transformers, and is the cutting-edge of pretraining languages for NLP AI tasks. BERT is a project of Google Research, and you can learn more about it on the BERT project page. To bring this diversion back to practical AI terms: BERT opens up all sorts of useful, practical NLP tasks, performed in an efficient manner that’s doable on a mobile device (e.g., the sort of device for which you might use Swift to write).

Tip

The academic paper that introduced BERT to the world, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, is also a great place to start learning about BERT.

The most accessible, useful, practical NLP task that we recommend starting your exploration of BERT with is question answering. There’s a great dataset that you can pair with BERT in order to explore this: the Stanford Question Answering Dataset (SQuAD). It’s full of things like this:

  • TEXT: Seismologists can use the arrival times of seismic waves in reverse to image the interior of the Earth. Early advances in this field showed the existence of a liquid outer core (where shear waves were not able to propagate) and a dense solid inner core. These advances led to the development of a layered model of the Earth, with a crust and lithosphere on top, the mantle below (separated within itself by seismic discontinuities at 410 and 660 kilometers), and the outer core and inner core below that. More recently, seismologists have been able to create detailed images of wave speeds inside the earth in the same way a doctor images a body in a CT scan. These images have led to a much more detailed view of the interior of the Earth, and have replaced the simplified layered model with a much more dynamic model.

  • QUESTION: What types of waves do seismologists use to image the interior of the Earth?

  • ANSWER: Seismic waves.

Apple actually makes BERT available for download as a CoreML model from its models site. Check it out and see what you can do with it!

Additionally, Apple has released a demo app that makes use of the BERT CoreML model, which you can download the source code to and try out.

A team from a “social artificial intelligence” startup (with which we have zero affiliation) has also done the hard work of making BERT work with iOS and CoreML (and appears to be the source of Apple’s provided CoreML version of BERT). You can find their work on GitHub. You can see an example of BERT working in a Swift iOS app using CoreML in Figures 6-18 and 6-19.

You might also be interested in generating text, which we introduce in our “Task: Sentence Generation”. In the next chapter we’ll look at motion.

pais 0619
Figure 6-18. BERT, working on iOS, using Swift and CoreML
pais 0620
Figure 6-19. BERT, working on iOS, using Swift and CoreML