Android: Getting Started with Databases and ContentProviders Part 1

Clone the code from GitHub

Android provides powerful database functionality built on SQLite. If you’re familiar with SQL from web development, you’ll be very comfortable with SQLite. However, what you might find is that it’s very easy to deal directly with the database (especially if you’re coming from a framework like Rails or Symfony).

Whatever language or platform you’re using, it’s always good practice to build an abstraction layer between your app’s code and its database. In this tutorial miniseries, we’ll walk through building a simple app that uses a database abstraction class, MyDatabaseHelper to manage querying, inserting, updating and deleting records from a simple database. In future posts, we’ll look at dealing with multiple tables and joins, and finally explore Android’s own ContentProvider API to abstract our database in a standard way.

Getting started

Start with a new Android application project (I recommend trying out the new Android Studio preview, especially if you’re just starting out with Android), and add the following files:

// src/com/example/MyActivity.java
public class MyActivity extends Activity {
  private MyDatabaseHelper mDatabaseHelper;
 
  public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
 
    mDatabaseHelper = new MyDatabaseHelper(this);
  }
}
// src/com/example/MyDatabaseHelper.java
public class MyDatabaseHelper extends SQLiteOpenHelper {
  public static final String TABLE_USERS = "users";
  public static final String COL_ID = BaseColumns._ID;
  public static final String COL_NAME = "name";
  public static final String COL_EMAIL = "email";
  public static final String COL_DOB = "date_of_birth";
  private static final String DATABASE_NAME = "my_app.db";
  private static final int DATABASE_VERSION = 1;
 
  public MyDatabaseHelper(Context context) {
    super(context, DATABASE_NAME, null, DATABASE_VERSION);
  }
 
  @Override
  public void onCreate(SQLiteDatabase db) {
    db.execSQL("CREATE TABLE " + TABLE_USERS + " ("
        + COL_ID + " INTEGER PRIMARY KEY AUTOINCREMENT,"
        + COL_NAME + " TEXT NOT NULL,"
        + COL_EMAIL + " TEXT,"
        + COL_DOB + " INTEGER"
        + ");");
  }
 
  @Override
  public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
    db.execSQL("DROP TABLE IF EXISTS " + TABLE_USERS + ";");
    onCreate(db);
  }
 
  public long insert(String tableName, ContentValues values) {
    return getWritableDatabase().insert(tableName, null, values);
  }
 
  public int update(String tableName, long id, ContentValues values) {
    String selection = COL_ID + " = ?";
    String[] selectionArgs = {String.valueOf(id)};
 
    return getWritableDatabase().update(tableName, values, selection, selectionArgs);
  }
 
  public int delete(String tableName, long id) {
    String selection = COL_ID + " = ?";
    String[] selectionArgs = {String.valueOf(id)};
 
    return getWritableDatabase().delete(tableName, selection, selectionArgs);
  }
}

The advantage of this approach is that MyDatabaseHelper becomes a single point of access to the raw database (following good DRY principles). It also provides the opportunity to validate data before it is written to the database. For example, we have added a NOT NULL constraint on the name column. If we tried to save a record to the database without setting a value for name, our app would crash with a SQLiteConstraintException. Let’s demonstrate this by adding a few users in our Activity, but leaving one of the names blank:

// src/com/example/MyActivity.java
  public void onCreate(Bundle savedInstanceState) {
    // ...    
    addUser(null, null, 0);
    addUser("Joe User", "joe@example.com", 0);
    addUser("Mary Jones", "mary@example.com", 0);
    addUser("Sue Bloggs", "sue@example.com", 0);
  }
 
 private void addUser(String name, String email, long dateOfBirthMillis) {
    ContentValues values = new ContentValues();
    values.put(MyDatabaseHelper.COL_NAME, name);
 
    if (email != null) {
      values.put(MyDatabaseHelper.COL_EMAIL, email);
    }
 
    if (dateOfBirthMillis != 0) {
      values.put(MyDatabaseHelper.COL_DOB, dateOfBirthMillis);
    }
 
    mDatabaseHelper.insert(MyDatabaseHelper.TABLE_USERS, values);
  }

Running this code, your app will crash with the SQLiteConstraintException we expected:

  ERROR/SQLiteDatabase(19185): Error inserting name=null
      android.database.sqlite.SQLiteConstraintException: users.name may not be NULL (code 19)
      at android.database.sqlite.SQLiteConnection.nativeExecuteForLastInsertedRowId(Native Method)
      at android.database.sqlite.SQLiteConnection.executeForLastInsertedRowId(SQLiteConnection.java:775)
      at android.database.sqlite.SQLiteSession.executeForLastInsertedRowId(SQLiteSession.java:788)
      ....

Adding Validations

Using our abstract Database helper class though, we can protect against this and handle the error much more elegantly. To do this, we’ll add some simple validations at the appropriate points in MyDatabaseHelper:

// src/com/example/MyDatabaseHelper.java
  public long insert(String tableName, ContentValues values) throws NotValidException {
    validate(values);
 
    return getWritableDatabase().insert(tableName, null, values);
  }
 
  public int update(String tableName, long id, ContentValues values) throws NotValidException {
    validate(values);
 
    String selection = COL_ID + " = ?";
    String[] selectionArgs = {String.valueOf(id)};
 
    return getWritableDatabase().update(tableName, values, selection, selectionArgs);
  }
 
  // ...
 
  protected void validate(ContentValues values) throws NotValidException {
    if (!values.containsKey(COL_NAME) || values.getAsString(COL_NAME) == null || values.getAsString(COL_NAME).isEmpty()) {
      throw new NotValidException("User name must be set");
    }
  }
 
  public static class NotValidException extends Throwable {
    public NotValidException(String msg) {
      super(msg);
    }
  }

Back in MyActivity, wrap the call to insert() in a try...catch block to capture the NotValidException and show a message to the user:

// src/com/example/MyActivity.java 
  private void addUser(String name, String email, long dateOfBirthMillis) {
    // ...  
    try {
      mDatabaseHelper.insert(MyDatabaseHelper.TABLE_USERS, values);
    } catch (MyDatabaseHelper.NotValidException e) {
      Toast.makeText(this, e.getMessage(), Toast.LENGTH_SHORT).show();
    }
  }

Now if you run your app, it will gracefully handle the error, and continue adding the other user records to the local database.

Querying and Displaying Records

At the moment, we can’t see our data though. We’ll add a quick ListView to MyActivity to show the list of user’s. If one doesn’t already exist, create a new layout resource file in your project: res/layouts/my_activity.xml and add a ListView to fill the screen:

// res/layout/my_activity.xml
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
              android:orientation="vertical"
              android:layout_width="match_parent"
              android:layout_height="match_parent">
 
    <ListView
        android:layout_width="fill_parent"
        android:layout_height="match_parent"
        android:id="@+id/listView"/>
</LinearLayout>

In MyActivity, make sure that you set the content view to be the new my_activity.xml layout:

// src/com/example/MyActivity.java
  public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.my_activity);
 
    // ...
  }

Next, create a SimpleCursorAdapter to retrieve data from the database and display it in our new ListView:

// src/com/example/MyActivity.java
  public void onCreate(Bundle savedInstanceState) {
    // ...
 
    Cursor c = mDatabaseHelper.query(MyDatabaseHelper.TABLE_USERS, MyDatabaseHelper.COL_NAME);
    String[] from = new String[]{MyDatabaseHelper.COL_NAME, MyDatabaseHelper.COL_EMAIL};
    int[] to = { android.R.id.text1, android.R.id.text2 };
 
    SimpleCursorAdapter adapter = new SimpleCursorAdapter(this, android.R.layout.simple_list_item_2, c, from, to, 0);
 
    ListView listView = (ListView) findViewById(R.id.listView);
    listView.setAdapter(adapter);
 
    // ...
  }

Finally, you’ll see that we’ve called a new method, query(), on our database helper. The parameters specify the table to query, and how to sort results. This will provide a simple way to retrieve the records from our database, wrapping the SQLiteOpenHelper‘s own query() method. Add the new method to MyDatabaseHelper:

 // src/com/example/MyDatabaseHelper.java
  public Cursor query(String tableName, String orderedBy) {
    String[] projection = {COL_ID, COL_NAME, COL_EMAIL, COL_DOB};
    return getReadableDatabase().query(tableName, projection, null, null, null, null, orderedBy);
  }

Of course, we could add other parameters such as search terms to our query(), making it very powerful – and again providing a single point of contact for querying our database.

Running the code, you’ll now see the list of names and email addresses as people are added to the database. Note that every time you re-run the app, another set of records will be added as the calls to addUser() are in the activity’s onCreate() method.

Screenshot of App Running

As you can see, it’s relatively easy to build a powerful abstraction layer between your database and application code, and there are huge advantages in doing so. As your app grows, you’ll start to build a library of common methods that will save you time and make your code more robust.

Coming Next

In the next tutorial, we’ll add another table to our database and adapt our database helper class to handle multiple tables and joins. In the final part, we’ll look at Android’s ContentProvider API, which provide a standard way to abstracting and manage access to your your app’s raw database.

Thanks for reading! If you’ve liked this tutorial, please let me know by leaving your comments and feedback on Twitter or Google+.

Join the free Plymouth Software newsletter today to get news, tutorials, hints and tips delivered straight to your inbox.

Rails 3: Storing Model Metadata Attributes with ActiveRecord::Store

I recently discovered the excellent ActiveRecord store method whilst researching the best-practice for storing optional, flexible metadata against a record. store lets you keep simple key/value data into a single text column on your model.

By declaring stored attributes on your model, ActiveRecord will automatically generate the appropriate setter/accessor methods, and validations work just as you'd expect.

# db/migrate/create_cars.rb
# ...
create_table do |t|
  t.references :model
  t.references :manufacturer
  t.text :metadata # Note metadata is just a text column
 
  t.timestamps
end
 
# app/models/car.rb
class Car &lt; ActiveRecord::Base
  belongs_to :model
  belongs_to :manufacturer
 
  # Manufacturer and Model are 'real', database-backed attributes
  attr_accessor :model_id, :manufacturer_id, :colour, :size, :notes, :product_url
 
  store :metadata, :accessors => [:colour, :size, :notes, :product_url]
 
  # Database-backed attributes
  validates :model, :presence, :presence => true
  validates :manufacturer, :presence => true
 
  # Metadata stored attributes
  validates :colour, :presence => true
  validates :size, :presence => true, :inclusion => { :in => %w(small medium large) }
  validates :product_url, :format => { :with => /^(https?:\/\/)?([\da-z\.-]+)\.([a-z\.]{2,6})([\/\w \.-]*)*\/?$/ }
end

Note that the metadata is just a text column. Rails will automatically serialise the hash into this column.

With our Car model defined, and its metadata attributes declared with store, we can use them in just the same way as normal, database-backed attributes:

car = Car.new
car.model = CarModel.first
car.manufacturer = Manufacturer.first
car.colour = "Red"
car.size = "medium"
car.product_url = "http://www.example.com/"
 
car.save
# => true
 
car.colour
# => "red"
 
car.size
# => "medium"

Pros and Cons

The biggest advantage to this type is store is it provides a flexible data schema. This is perfect for storing non-indexed metadata about a model, and a use-case that often crops up when building apps. It provides a compromise between common relational databases (such as MySQL and PostgreSQL) and the flexibility of NoSQL databases such as MongoDB and CouchDB. In practical use, it means attributes can quickly be added to a model without the need to perform any migrations on the database schema.

A possible disadvantage is that stored attributes can not be indexed or used in queries (as they have no corresponding database column). For example, you could not call Car.where(:colour => "red"). However, as Garry Tan points out in his post, if this becomes a requirement in the future, you could always add a database-column to the schema at a later date, and just move the data at that time.

Helpful post? Let me know on Twitter.

References

  1. http://axonflux.com/one-of-my-favorite-additions-to-rails-3-activ

Welcoming 2013 – Goals for the New Year

2012 was the year I said farewell to my twenties, celebrated the birth of our beautiful baby daughter, and started to see some real growth in my business. Whilst blog posts were evidently sparse, it has been an absolutely incredible year, both personally and professionally. So much has happened that the past 12 months seem to have flown by.

In this post, I’ll document a brief summary of what’s happened, and offer my thoughts and goals for the new year.

IMG_20120619_202506

The Past Year

In January, I moved Plymouth Software from the incubating LaunchPad to its own offices on Tamar Science Park Scott Building. The business has grown its client base and income over the year, and whilst by no means enough to let me take a mini-retirement yet, feels a lot more solid than 12 months ago.

As well as winning several new clients (commercial, academic and charitable), I also expanded the company’s service offer to include development of native iOS applications, and I am currently engaged developing an iPhone app due for release in February 2013.

In June, I ran the Plymouth Half Marathon, shaving a couple of minutes off my 2011 time. Thanks to the generosity of friends, family, colleagues, and everyone else who donated, I managed to raise over £600 for Macmillan Cancer Support.

We travelled to Spain, Jersey, Scotland, several cities in Northern England, lots of spots around Devon and Cornwall; and of course, drank plenty of tea.

Of course, the highlight of the year occurred in August, when Claire and I celebrated the arrival of our baby daughter. Becoming a Dad really puts a new perspective on things, which has inevitably changed my thinking for what I want to achieve in the next 12 months.

2013: The Goals

Produce More, Consume Less.

With our amazing new addition, running and building Plymouth Software, and generally trying to balance life and business, my time seems to have become incredibly limited. This is evident by the sheer lack of posts here. I realise though that this is just an excuse – it’s all too easy to be suckered into the endless river of social media streams, news feeds, blog , etc. and procrastinatory opportunities that present themselves when you run your own business. My first goal then is to focus on producing rather than consuming. The Internet is endless in its ability to offer more, so it’s better to try to output (quality) content rather than read it.

Goal: Publish at least one relatively in-depth blog post to this site each month. Write content for products (see goals below).

Focus

Lack of focus really hit me this year when I realised just how many task and project management apps I’d signed up for in an effort to become more organised. The time spent trying out what are essentially list apps could have been put to far better use. Again, the endless distraction of the Internet was the cause, offering an escape from dealing with code problems and producing.

Back in 2011, I found the Pomodoro technique incredibly useful (using the excellent Pomodroido app), but for some reason I stopped. The resulting lack of focus has led to frustration.

Goal: Start using the Pomorodo technique again to focus on work that generates value. I’ve also signed up to Freckle for the quickest way to see where my time is being spent.

Sidenote: For task management, I settled on Wunderlist. With the release of version 2, they’ve added the absolutely vital Reminders and Repitition to tasks, which were essential. Wunderlist is also multi-platform, and now has a nice native Android app.

Balance

As part of developing Plymouth Software’s business model, one of my aims is to ensure a good work-life balance. As more client work has come in and deadlines approach, I’ve felt more of a drag to do just a few minutes in evenings and weekends.

Goal: Hit the reset button to keep work at the office. Establish business hours with myself and clients, and stick to them.

Sidenote: According to others’ experiences, reducing time available to work talks should actually help to improve my focus and efficiency.

Products

When it’s your own, I don’t believe selling time is a viable way to build a sustainable business – irrespective of your rate. In order to grow Plymouth Software financially, it needs to a fundamental change away from the freelance model (selling my time) to a business model, sustainable through the sale products, or services which can be packaged as products, that begin to remove my time from the equation.

This has been the aim since day one. I’ve tried to figure this out before, but could never make the break away from selling time. The freelance model also provided the quickest way to generate some income and build capital, which is essential for a bootstrapped business. Now, though, in order to grow the company (both financially and taking on staff), this approach needs to change.

Reading about very talented people in similar situations have made the switch inspired me to get thinking, and taking Brennan Dunn’s excellent Consultancy Masterclass opened my eyes to a new way of productising my services, as well as offering products that are not a traditional part of the business, such as eBooks, code plugins, and workshops.

Goal: Schedule time and funds to develop revenue-generating assets (more details on the first of those in the next post).

Finance

To my mind, self-employment – having the ability to quickly adapt to situations and generate new streams of income – has always been far more stable than being an employee, where your income is firmly in the hands of somebody else. The real advantage of working for yourself is the freedom to be in control of my own future, as well as the sheer enjoyment that comes with building up a business.

Plymouth Software has so far granted an income and turned a profit in Year 1. However, in 2013, with lots of new responsibilities, I would like to at least double the turnover that the company has made in its first year (as a limited company).

This should be easily achievable by fulfilling some of the goals above, as well as following in the footsteps of bigger consultancy firms from whom I take inspiration. For example, I hadn’t come across weekly billing until taking Brennan’s Consultancy Masterclass course, but it seems to be quite common in software development shops, particularly those practicing agile techniques.

Goal: Double the company’s turnover for Year 2. Acquire at least 4 retainer-style contracts for clients. Set value-based standard rates, and move to a weekly billing model for project/consultancy jobs.

Brand / Marketing

I’ll be the first to admit the business must improve its marketing output. The company website is functional, but the portfolio is already outdated, and doesn’t much promote the core service offering of the company. Along with changes the underlying business model above, I’d like to take the opportunity to refine the company brand itself.

One thing I’ve recognised is that the business actually offers clients much more than simple code-and-deliver development services. As I’ve taken on more involved projects, I’ve found myself consulting on a range of subjects, from business development and marketing, through to experience design and technical environment considerations.

As I move the company away from a freelance model, I need to ensure that the brand reflects the full scope of what the company offers.

Goal: Reduce friction of updating company website by moving to more flexible content management system. Rewrite marketing copy to focus on full range of services offered. Send at least one email newsletter per month. Write one white-paper on mobile apps, to be distributed though the company web site.

Fitness

In both 2011 and 2012 I ran the Plymouth Half Marathon, and the fitness that came with running was great. However, with no running goal for 2013 – and a lot of sleepless nights since August – my running has taken a back seat. In 2013, I’d like to rectify this by committing to get back into running, as well as taking advantage of January membership discounts for the local gym and swimming pool.

Goal: Get back into running at least once a week, minimum of 3 miles per run (road or gym). Gym or Swim session twice a week.

Final Thoughts – The Overall Goal

Hopefully, as we move to 2014, I’ll be able to look back over this post and check off all those goals.

I’m sure though, like any plan, they will change a little throughout the course of 2013, but it feels good to have committed to writing where I want to be. Ultimately, the most important goal (and that which has influenced all others) is to provide and spend as much quality time with our daughter as she grows up.

2013 is going to be a great – Happy New Year!

Running Plymouth Half Marathon – Let’s Raise £1k for Macmillan Cancer Support

Last year, I successfully completed my first Plymouth Half Marathon and raised for local charity Jeremiah’s Journey.

In memory of a close friend, Pat Mourton, who sadly passed away from cancer in August 2011, I am running the Plymouth Half Marathon again on the 3 June 2012. My goal is to raise at least £1k for Macmillan Cancer Support, who gave incredible and ongoing support to both Pat and her family throughout her battles with cancer.

You can be part of helping me to reach (or beat!) my target, and raise as much as possible for Macmillan by donating on my Just Giving page at http://www.justgiving.com/chrisblunt-plymouth2012.

If you can’t donate today, then please help spread the word by posting the link on your favourite social networks.

Thank you!

In 1996, Pat Mourton was diagnosed with breast cancer. I was lucky enough to meet Pat through her son Richard and husband Alan, during my first days at secondary school. They quickly became very close friends.

Pat was well known for her work in the local community, and everyone who was fortunate enough to meet her couldn’t help but be inspired by her spirit, determination, and (perhaps most of all) her unfaltering sense of humour.

In the nearly 2 decades she lived with the disease, Pat fought and beat the breast cancer, and bravely endured bone cancer and later metastatic liver cancer. She ran countless fundraising events for the charities that would become close to her heart.

Sadly, Pat lost her battle with cancer on 15 August 2011.

Macmillan nurses provide exceptional support and care not only to cancer sufferers, but their families who face this horrible disease. Macmillan nurses truly went above-and-beyond the call of duty for Pat, Alan and Richard.

In memory of Pat, and to help Macmillan carry on this tireless work, I’m running the Plymouth Half Marathon for a second time on 3 June 2012 and need to raise £1000 for Macmillan.

Please donate whatever you can to help me reach – and smash – my target of raising £1000 for Macmillan Cancer Support so that other people can continue to receive the support that was so important to Pat, Alan and Richard.

Please take a moment to send this page to everyone you know to help support those who are suffering with cancer.

Thank you,
Chris

Testing Rendered Views with RSpec2, Capybara and Rails 3.

While view tests are brittle, easily breaking when the design of a page changes, they are undoubtedly handy for checking the important parts of a page are rendered. I usually check for page titles, model attributes/tables or forms, and footer element, and so on.

Capybara is a great DSL for testing a page’s rendered output, but unfortunately the Capybara matchers are not available in view or helper specs according to the rspec-rails documentation.

Thanks to this answer on Stack Overflow, it’s possible to load a page’s rendered content into a controller spec, and test it using the Capybara matchers.

I’m posting some example code here based on the answer for future reference, and to help anyone struggling to use Capybara with Rspec 2:

# Gemfile
group :test do
  gem 'capybara'
  gem 'rspec-rails'
end
# spec/controllers/users_controller_spec.rb
require 'spec_helper'
 
describe UsersController do
  # Render views and load the rendered content into Capybara for matching
  # See: http://stackoverflow.com/questions/4706370/rspec-view-testing-with-capybara-and-rails3/4773050#4773050#
  render_views
  let(:page) { Capybara::Node::Simple.new(@response.body) }
 
  describe "GET 'index'" do
    before { get :index }
 
    it { should respond_with(:success) }
    it { should assign_to(:users).with_kind_of(Array) }
    it { should render_template(:index) }
 
    # Test the rendered view using Capybara matchers
    it { page.should have_selector("h2") }
    it { page.should have_content("All Users") }
    it { page.should have_selector("table#users") }
    # ... add more tests ...
    it { page.should have_selector("div#footer" }
  end
end

Helpful post? Let me know by commenting on Twitter.