Getting Answers for Core ML deployment from my own Book

I was working today in the deployment of a small neural network model prototype converted to Core ML to be used in an iPhone app.

I was trying to find the best way to get things to work and then it occurred to me I had solved a similar issue before… where‽ when‽ aha!

The answer was actually in my Advanced Data Science and Analytics with Python.

MacOS – No Floating Thumbnail when taking a screenshot

Have you tried taking a screenshot in your Mac and are annoyed at having to wait for the floating thumbnail – in other words you wait for 5 seconds before the screenshot becomes a file? Well here you can find out how to get rid of that.

Follow these steps:

1) Type CMD + SHIFT + 5
2) Click OPTIONS
3) Uncheck “Show Floating Thumbnail”
4) Et voilà!

See the screenshot above!

Apple Developer Support

It is great to see all the support that Apple Developers get in terms of tools, ecosystem, community and more.

Apple_support

For starters the Developer Support portal has a ton of information for the new comer as well as for the more expert of experts. Including guides and documentation for tools such as Xcode as well as information for developing software for MacOS and iOS.

Information about Design is available in the same place, including Human Interface Guidelines, Fonts (including downloads for San Francisco!) and information about accessibility and localisation.

Information about new tools and updates such as the latest about Swift, and SwiftUI can be easily found. And testing your apps with the help of tools such as TestFlight makes things so much easier.

File Encoding with the Command Line – Determining and Converting

With the changes that Python 3 has brought to bear in terms of dealing with character encodings, I have written before some tips that I use on my day to day work. It is sometimes useful to determine the character encoding of a files at a much earlier stage. The command line is a perfect tool to help us with these issues. 

The basic syntax you need is the following one:

$ file -I filename

Furthermore, you can even use the command line to convert the encoding of a file into another one. The syntax is as follows:

$ iconv -f encoding_source -t encoding_target filename

For instance if you needed to convert an ISO88592 file called input.txt into UTF8 you can use the following line:

$ iconv -f iso-8859-1 -t utf-8 < input.txt > output.txt

If you want to check a list of know coded characters that you can handle with this command simply type:

$ iconv --list

Et voilà!

 

Magic Mouse – Secondary Click Not Working

I have recently taken Mojave for a spin and I am really happy with the changes in the new OS. I know it is merely eye-candy, but I really like the dark theme. Things have been working well, but I came across a nagging issue with my MagicMouse:

For some reason the secondary click would simply not work. I had made sure the settings were enabled by making sure that the “Secondary Click” option was ticked (see screenshot below). I tried ticking it on and off, restarting the machine, deleting the mouse and reconnecting it… nothing had worked…

Finally I decided to take a look at some of the plist files and here is my solution to this problem:

    1. Go to the ~/Library/Preferences/ directory
    2. Delete the following files:
      com.apple.AppleMultitouchMouse.plist
      com.apple.driver.AppleBluetoothMultitouch.mouse.plist
      
      Restart the machine

Et voilà!

Persistent “Previous Recipients” in Mac Mail

Hello everyone! I am very pleased to take a question from John who got in touch with Quantum Tunnel using the form here. John’s favourite scientist is Einstein and his question is as follows:

In Mac mail I cannot delete unwanted email addresses. I have done the routine of deleting all addresses from the previous receiptant list, but when starting a new email unwanted addresses appear.. Any help is appreciated. Thanks, John

John is referring to the solution I provided in this earlier post. Sadly, the list of his lucky friends/colleagues/family (delete as appropriate) he has email recently persists even after clearing the “Previous Recipients” as explained in the post before.

There may be a way to force the clearing of these persistent email address:

  • Quit Mail and Address Book (in case the latter is open)
  • Open a terminal and type the following command:
    • `rm ~/Library/Application Support/AddressBook/MailRecents-v4.abcdmr`
  • Log out and back in again
  • Start Mail
  • You may have to clear the “Previous Recipients” list as per the post mentioned above

You should now be able to clear the list. And… In case you were wondering, the file we deleted should be created afresh to start accumulating new “recent recipients” (yay!)

Et voilà!

CoreML – Boston Model: The Complete App

Look how far we have come… We started this series by looking at what CoreML is and made sure that our environment was suitable. We decided to use linear regression as our model, and chose to use the Boston Price dataset in our exploration for this implementation. We built our model using Python and created our .mlmodel object and had a quick exploration of the model’s properties. We then started to build our app using Xcode (see Part 1, Part 2 and Part 3). In this final part we are going to take the .mlmodel and include it in out Xcode project, we will then use the inputs selected from out picker and calculate a prediction (based on our model) to be displayed to the user. Are you ready? Nu kör vi!

Let us start by adding the .mlmodel we created earlier on so that it is an available resource in our project. Open your Xcode project and locate your PriceBoston.mlmodel file. From the menu on the left-hand side select the “BostonPricer” folder. At the bottom of the window you will see a + sign, click on it and select “New Groups”. This will create a sub-folder within “BostonPricer”. Select the new folder and hit the return key, this will let you rename the folder to something more useful. In this case I am going to call this folder “Resources”.

Open Finder and navigate to the location of your BostonPricer.mlmodel. Click and drag the file inside the “Resources” folder we just created. This will open a dialogue box asking for some options for adding this file to your project. I selected the “Create Folder References” and left the rest as it was shown by default. After hitting “Finish” you will see your model now being part of your project. Let’s now go the code in ViewController and make some needed changes.  The first one is to tell our project that we are going to need the powers of the CoreML framework. At the top of the file, locate a line of code that imports UIKit, right below it type the following:

import CoreML

Inside the definition of the ViewController class, let us define a constant to reference the model. Look for the definitions of the crimeData and roomData constants and nearby them type the following:

let model = PriceBoston()

You will see that when you start typing the name of the model, Xcode will suggest the right name as it knows about the existence of the model as part of its resources, neat!

We need to make some changes to the getPrediction()function we created in the last post. Go to the function and look for place where we pick the values of crime and rooms and right after that write the following:

guard let priceBostonOutput = try? model.prediction(
            crime:crime,
            rooms: Double(rooms)
            ) else {
                fatalError("Unexpected runtime error.")
        }

You may get a warning telling you that the constant priceBostonOutput was defined but not used. Don’t worry, we will indeed use it in a little while. Just a couple of words about this piece of code, you will see that we are using the prediction method defined in the model and that we are passing the two input parameters that the model expects, namely crime and rooms. We are wrapping this call to the prediction method around a try statement so that we can catch any exceptions. This is where we are implementing our CoreML mode!!! Isn’t that cool‽

We are not done yet though; remember that we have that warning from Xcode about using the model. Looking at the properties of the model, we can see that we also have an output attribute called price. This is the prediction we are looking for and the one we would like to display. Out of the box it may have a lot of decimal figures, and it is never a good practice to display those to the user (although they are important in precision terms…). Also, with Swift’s strong typing we would have to typecast the double returned by the model into a string that can be printed. So, let us prepare some code to format the predicted price. At the top of the ViewController class, find the place where we defined the constants crimeData and roomData. Below them type the following code:

let priceFormat: NumberFormatter = {
        let formatting = NumberFormatter()
        formatting.numberStyle = .currency
        formatting.maximumFractionDigits = 2
        formatting.locale = Locale(identifier: "en_US")
        return formatting
    }()

We are defining a format that will show a number as currency in US dollars with two decimal figures. We can now pass our predicted price to this formatter and assign it to a new constant for future reference. Below the code where the getPrediction function was defined, write the following:

let priceText = priceFormat.string(from: NSNumber(value:
            priceBostonOutput.price))

Now we have a nicely formatted string that can be used in the display. Let us change the message that we are asking our app to show when pressing the button:

let message = "The predicted price (in $1,000s) is " + priceText!

We are done! Launch your app simulator, select a couple of values from the picker and hit the “Calculate Prediction” button… Et voilà, we have completed our first implementation of a CoreML model in a working app.

There are many more things that we can do to improve the app. For instance, we can impose some constraints on the position of the different elements shown in the screen so that we can deploy the application in the various screen sizes offered by Apple devices. Improve the design and usability of the app and designing appropriate icons for the app (in various sizes). For the time being, I will leave some of those tasks for later. In the meantime you can take a look at the final code in my github site here.

Enjoy and do keep in touch, I would love to hear if you have found this series useful.

 

CoreML – iOS Implementation for the Boston Model (part 2) – Filling the Picker

Right! Where were we? Yes, last time we put together a skeleton for the CoreML Boston Model application that will take two inputs (crime rate and number of rooms) and provide a prediction of the price of a Boston property (yes, based on somewhat all prices…). We are making use of three three labels, one picker and one button.

Let us start creating variables to hold the potential values for the input variables. We will do this in the ViewController by selecting this file from the left-hand side menu:

 

 

 

 

 

 

 

Inside the ViewController class definition enter the following variable assignments:

let crimeData = Array(stride(from: 0.1, through: 0.3, by: 0.01))
let roomData = Array(4...9)

These values are informed by the data exploration we carried out in an earlier post. We are going to use the arrays defined above to populate the values that will be shown in our picker. For this we need to define a data source for the picker and make sure that there are two components to choose values from.

Before we do any of that we need to connect the view from our storyboard to the code, in particular we need to create outlets for the picker and for the button. Select the Main.storyboard from the menu in the left-hand side. With the Main.storyboard in view, in the top right-hand corner of Xcode you will see a button with an icon that has two intersecting circles, click on that icon. you will now see the storyboard side-by-side with the code. While pressing the Control key, select the picker by clicking on it; without letting go drag into the code window (you will see an arrow appear as you drag):

 

 

You will se a dialogue window where you can now enter a name for the element in your Storyboard. In this case I am calling my picker inputPicker, as shown in the figure on the left. After pressing the “connect” button a new line of code appears and you will see a small circle on top of the code line number indicating that a connection with the Storyboard has been made. Do the same for the button and call it predictButton.

 

 

In order to make our life a little bit easier, we are going to bundle together the input values. At the bottom of the ViewController code write the following:

enum inputPredictor: Int {
    case crime = 0
    case rooms
}

We have define an object called inputPredictor that will hold the values of for crime and rooms. In turn we will use this object to populate the picker as follows: In the same ViewController file, after the class definition that is provided in the project by  default we are going to write an extension for the data source. Write the following code:

extension ViewController: UIPickerViewDataSource {

    func numberOfComponents(in pickerView: UIPickerView) -> Int {
        return 2
    }

    func pickerView(_ pickerView: UIPickerView,
                    numberOfRowsInComponent component: Int) -> Int {
        guard let inputVals = inputPredictor(rawValue: component) else {
            fatalError("No predictor for component")
        }

        switch inputVals {
        case .crime:
            return crimeData.count
        case .rooms:
            return roomData.count
        }
    }
}

With the function numberOfComponents we are indicating that we want to have 2 components in this view. Notice that inside the pickerView function we are creating a constant inputVals defined by the values from inputPredictor.  So far we have indicated where the values for the picker come from, but we have not delegated the actions that can be taken with those values, namely displaying them and picking them (after all, this element is a picker!) so that we can use the values elsewhere. If you were to execute this app, you will see an empty picker…

OK, so what we need to do is create the UIPickerViewDelegate, and we do this by entering the following code right under the previous snippet:

extension ViewController: UIPickerViewDelegate {
    func pickerView(_ pickerView: UIPickerView, titleForRow row: Int,
                    forComponent component: Int) -> String? {
        guard let inputVals = inputPredictor(rawValue: component) else {
            fatalError("No predictor for component")
        }

        switch inputVals {
        case .crime:
            return String(crimeData[row])
        case .rooms:
            return String(roomData[row])
        }
    }

    func pickerView(_ pickerView: UIPickerView, didSelectRow row: Int,
                    inComponent component: Int) {
        guard let inputVals = inputPredictor(rawValue: component) else {
            fatalError("No predictor for component")
        }

        switch inputVals {
        case .crime:
            print(String(crimeData[row]))
        case .rooms:
            print(String(roomData[row]))
        }


    }
}

In the first function we are defining what values are supposed to be shown for the titleForRow in the picker, and we do this for each of the two elements we have, i.e. crime and rooms. In the second function we are defining what happens when we didSelectRow, in other words select the value that is being shown by each of the two elements in the picker. Not too bad, right?

Well, if you were to run this application you will still see no change in the picker… Why is that? The answer is that we need to let the application know what needs to be show when the elements load. Go back to the top of the code (around line 20 or so) below the code lines that defined the outlets for the picker and the button. There write the following code:

override func viewDidLoad() {
    super.viewDidLoad()
    // Picker data source and delegate
    inputPicker.dataSource = self
    inputPicker.delegate = self
}

OK, we can now run the application: On the top left-hand side of the Xcode window you will see a play button; clicking on it will launch the Simulator and you will be able to see your picker working. Go on, select a few values from each of the elements:

In the next post we will write code to activate the button to run a prediction using our CoreML model with the values selected from the picker and show the result to the user. Stay tuned!

You can look at the code (in development) in my github site here.

Core ML – Preparing the environment

Hello again! In preparation to training a model to be converted by Core ML to be used in an application, I would like to make sure we have a suitable environment to work on. One of the first things that came to my attention looking at the coreml module is the fact that it only supports Python 2! Yes, you read correctly, you will have to make sure you use Python 2.7 if you want to make this work. As you probably know, Python 2 will be retired in 2020, so I hope that Apple is considering in their development cycles. Python 3 is now finally supported! In the meantime you can see the countdown to Python 2’s retirement here, and thanks Python 2 for the many years of service…

Anyway, if you are a Python 2 3 user, then you are good to go. If on the other hand you have moved with the times you may need to make appropriate installations. I am using Anaconda (you may use your favourite distro) and I will be creating a conda environment (I’m calling it coreml) with Python 2.7 and some of the libraries I will be using:

> conda create --name coreml python=3 ipython jupyter scikit-learn

> conda activate coreml

(coreml) 
> pip install coremltools

I am sure there may be other modules that will be needed, and I will make appropriate installations (and additions to this post) as that becomes clearer.

You can get a look at Apple’s coremltools github repo here.

ADDITIONS: As I mentioned, there may have been other modules that needed installing in the new environment here is a list:

  • pandas
  • matplotlib
  • pillow