In 2016 Cloud Construct will be working with Windows Azure Media Services for a handful of our agency's clients. Windows Azure Media Services is a subset of the many features that Windows Azure provides, and is one that builds upon already existing services that are offered in the Windows Azure suite. For example, a person could roll their own media storage, encoding server, and streaming engine in Azure using BLOB storage and some virtual machines. But unless you need a very custom configuration then why would you? Azure Media Services offers such a rich feature set such as Indexing, Redundant Storage, Encoding, and Live Streaming. With all of these new features comes a big learning curve in order to become an expert at using these great Windows tools. I set out to find a way to easily store, manage, and query images in Windows Azure using Media Services but quickly realized that Media Services was not the tool for our needs.

Image management is something that can sound very simple at first pass, but can have sprawling engineering requirements as a product becomes more complex. Questions such as:

1. How can we quickly search and retrieve images?
2. How do we transfer and store images in the quickest and most cost effective way?
3. How can we be sure to store a quality version of each file so if it needs to be cropped or edited it can retain quality later?
4. What image formats will be support?
5. Do we need to store sizes for a variety of screen sizes? Desktop, mobile, tablet?
6. What are the disaster recovery scenarios we need to plan for? Geo redundant storage? Locally redundant?
7. Can we leverage a CDN to serve our images efficiently around the world?

These are just some of the questions that come up when thinking about managing an engineering project with an image media library. After investigating the offerings from Azure Media Services for images, I came to the conclusion that it did not make sense to use media services for my image storage project. Window's uses the term "media" very loosely and while it does support scenarios to upload images, but it is not designed for more robust storage, serving, and editing of large volumes of images. Windows Azure Media Services appears to be strictly a solution for managing audio/video assets. Because of this I had to come up with an alternative strategy to manage image assets, easily search them, and allow for expansion to process them.

The solution I came up with was to use a combination of Azure BLOB storage, DocumentDB, and Azure Search, to easily store and manage large amounts of image media. Any relational or domain data can be kept in SQL Azure for referencial purposes, but we will use an embedded document store for all album and image metadata. I was able to bring a few great tutorials together from Microsoft to achieve the results I was looking for. The steps I took are outlined below.

Step 1 - Storage Setup

You will need to configure new accounts for Storage, DocumentDB, and Azure Search. These configurations should be completed using the "New" Azure portal since it has support for all the features we will require.

Configure Azure Storage: Add a new storage account in Azure and create a new container for your image assets labeled "assets". For now, just make this container public. In the future we would want to secure this if need be.

Lastly, you must configure an Azure Search account: Add a new Azure Search account and allow Azure to create this new resource for you. Once it is complete take note of the ServiceName and the API Key.

Step 2 - Development Environment Tools

Visual Studio 2015 Update 1 has some great new Azure tools. It is pretty easy for purposes of this walk-through to interact with the Azure services via the Cloud Explorer found in Visual Studio. We will need to upload an image to BLOB storage and push some documents into DocumentDB in order to get our demo to work.

Also, I found this tool called DocumentDBStudio to be very helpful in allowing me to manage DocumentDB data as well. You can download the latest code and just build it so you have an executable to run.

Step 3 - DocumentDB Code Setup

I setup a new ASP.NET MVC solution and basically followed this Microsoft tutorial. The tutoral instructs you to set up a new DocumentDB account in Azure and allow Azure to spin that up for you. Take note of the EndpointURI and either the Primary or Secondary key in the portal for your application to connect to the DocumentDB.

You will then be able to follow the remaining steps to add Nuget packages to your new project and also add the required code classes as well. If you follow this tutorial you should be able to get a running project going that allows you to very easily interact with DocumentDB. For purposes of this walk-through I want to create a project that is able to store image metadata and organize them by albums. In order to do this I had to modify some steps from this tutorial to fit my needs. Perform these steps prior to running the application for the first time because the code will create a new database if it does not exist.

1. Change the database name from Todos -> MediaLibrary and change the collection name from items -> Assets.

<add key="endpoint" value="enter the URI from the Keys blade of the Azure Portal"/>
<add key="authKey" value="enter the PRIMARY KEY, or the SECONDARY KEY, from the Keys blade of the Azure  Portal"/>
<add key="database" value="MediaLibrary"/>
<add key="collection" value="Assets"/>

2. Next in Step 4, change the class file name to "Image.cs" and the class name to "Image". An image will be associated to an Album by the "albumId" property. The property of blobStorageUrl will be the actual location of the image media we will place into BLOB storage for reference. The properties of the class should be changed to the following:

public class Image
{
 [JsonProperty(PropertyName = "id")]
 public string Id { get; set; }

 [JsonProperty(PropertyName = "assetType")]
 public string AssetType { get; set; }

 [JsonProperty(PropertyName = "albumId")]
 public string AlbumId { get; set; }

 [JsonProperty(PropertyName = "name")]
 public string Name { get; set; }

 [JsonProperty(PropertyName = "desc")]
 public string Description { get; set; }

 [JsonProperty(PropertyName = "blobStorageUrl")]
 public string BlobStorageUrl { get; set; }

 [JsonProperty(PropertyName = "isPublic")]
 public bool IsPublic { get; set; }  
}

3. Create another class file named "Album.cs" and the class name to "Album". An Album will be a logical grouping of images. The properties of the class should be set to the following:

public class Album
{
[JsonProperty(PropertyName = "id")]
 public string Id { get; set; }

 [JsonProperty(PropertyName = "assetType")]
 public string AssetType { get; set; }

 [JsonProperty(PropertyName = "name")]
 public string Name { get; set; }

 [JsonProperty(PropertyName = "desc")]
 public string Description { get; set; }

 [JsonProperty(PropertyName = "isPublic")]
 public bool IsPublic { get; set; }
}

4. Modify the controller that they have you create in Step 4, to match this. You will see we just modified the logic so that it works for our Album type. We use the AssetType property to query DocumentDB for the appropriate resources. There is nothing wrong with storing diffferent entities types in the same database collection. It is not analgous to a SQL table.

public class HomeController : Controller
 {

 public HomeController()
 {
 
 }

 public ActionResult Index()
 {

 var items = DocumentDBRepository<Album>.GetItems(d => d.AssetType == "Album");
 return View(items);
 }
 public ActionResult Create()
 {
 return View();
 }

 [HttpPost]
 [ValidateAntiForgeryToken]
 public async Task<ActionResult> Create([Bind(Include = "Id,Name,Description,IsPublic")] Album item)
 {
 if (ModelState.IsValid)
 {
 item.AssetType = "Album";
 await DocumentDBRepository<Album>.CreateItemAsync(item);
 return RedirectToAction("Index");
 }

 return View(item);
 }
 public ActionResult Edit(string id)
 {
 if (string.IsNullOrEmpty(id))
 {
 return new HttpStatusCodeResult(HttpStatusCode.BadRequest);
 }

 Album item = (Album)DocumentDBRepository<Album>.GetItem(d => d.Id == id);

 if (item == null)
 {
 return HttpNotFound();
 }

 return View(item);
 }

 [HttpPost]
 [ValidateAntiForgeryToken]
 public async Task<ActionResult> Edit([Bind(Include = "Id,Name,Description,IsPublic")] Album item)
 {
 if (ModelState.IsValid)
 {
 await DocumentDBRepository<Album>.UpdateItemAsync(item.Id, item);
 return RedirectToAction("Index");
 }

 return View(item);
 }
 }

5. You will want to create another controller to match this. This controller if used to manage the image assets.

public class ImagesController : Controller
 {

 public ImagesController()
 {
 }

 public ActionResult Index(string albumId)
 {
 var items = DocumentDBRepository<Image>.GetItems(d => d.AssetType == "Image" && d.AlbumId == albumId);
 ViewBag.AlbumId = albumId;
 return View(items);
 }

 public ActionResult Create(string albumId)
 {
 ViewBag.AlbumId = albumId;
 return View();
 }

 [HttpPost]
 [ValidateAntiForgeryToken]
 public async Task<ActionResult> Create([Bind(Include = "Id,AlbumId,Name,Description,BlobStorageUrl,IsPublic")] Image item)
 {
 if (ModelState.IsValid)
 {
 item.AssetType = "Image";
 await DocumentDBRepository<Image>.CreateItemAsync(item);
 return RedirectToAction("Index");
 }

 return View(item);
 }
 public ActionResult Edit(string id)
 {
 if (string.IsNullOrEmpty(id))
 {
 return new HttpStatusCodeResult(HttpStatusCode.BadRequest);
 }

 Image item = (Image)DocumentDBRepository<Image>.GetItem(d => d.Id == id);

 if (item == null)
 {
 return HttpNotFound();
 }

 return View(item);
 }

 [HttpPost]
 [ValidateAntiForgeryToken]
 public async Task<ActionResult> Edit([Bind(Include = "Id,AlbumId,Name,Description,BlobStorageUrl,IsPublic")] Image item)
 {
 if (ModelState.IsValid)
 {
 await DocumentDBRepository<Image>.UpdateItemAsync(item.Id, item);
 return RedirectToAction("Index");
 }

 return View(item);
 }
 }

6. You will need to create the corresponding views for these controller actions. You can simply duplicate the files the tutorial asks you to create and then you will want to modify the properties in the Listing, Edit, Create files to match the new Album and Image properties. Here is a snapshot of my MVC area I created. Disregard the "ImageSearch.cs" file, that will come into play later when we add Azure Search functionality.

7. Lastly, you will need to drop a couple hidden fields into the view files so "AlbumId" is persisted when browsing from the album listing page to the image listing page. This will help us properly add an image to an album and also list images associated with an album. The controller code is already above. Here are the view file changes :

• In Home/Index.cshtml add a new link next to "Edit" like so : @Html.ActionLink("Images", "Index", "Images", new { albumId = item.Id }, null)
• In Images/Index.cshtml add a link at the top like so : @Html.ActionLink("Create New", "Create", new { albumId = ViewBag.AlbumId })
• In Images/Index.cshtml set the rendering logic for BlobSotrageURL to be : <img src="@item.BlobStorageUrl" alt="@item.Name" />
• In Images/Create.cshtml add a hidden field like so : @Html.HiddenFor(model => model.AlbumId, new { value = ViewBag.AlbumId })
• In Images/Edit.cshtml add a hidden field like so : @Html.HiddenFor(model => model.AlbumId)

These modifications will allow us to keep the AlbumId persisted when working the Albums and Images. You don't need to let the user edit AssetType and this is accounted for in the Controller.

At this point you should be able to run the project and get a page that let's you list, create, and edit Albums. As well as click on Images associated with Albums and add more images to them. Here is what my Albums page looks like after adding my first album

You can see our properties are filled in and we can now either edit this album or check out the images associated with it. After clicking images, you will be brought to a screen that is empty because you have not created any images. Click your "Create New" link and add a new image giving a BLOB storage URL of existing test image you have. For me, I uploaded an image to my storage account in public container for demo purposes. My image listing page for this album looks like this:

You can see our properties are filled in and we're rendering our image properly that is associated in BLOB storage. Disregard the search box for now. We are now able to run a project that can easily interact with DocumentDB to store two asset type documents. Looking at DocumentDB storage using the Visual Studio Cloud Explorer we can see that there are now two documents in there as well.

The Image Document :

{
"id": "79e60b53-d5a3-4f0c-9158-ca348c6464f4",
"_rid": "xdhrAPJZGQAGAAAAAAAAAA==",
"_self": "dbs/xdhrAA==/colls/xdhrAPJZGQA=/docs/xdhrAPJZGQAGAAAAAAAAAA==/",
"_ts": 1451928629,
"_etag": "\"00001600-0000-0000-0000-568aac350000\"",
"assetType": "Image",
"albumId": "53ab666f-d537-4ceb-a6dd-72cd76d4e525",
"name": "Company",
"desc": "Our company logo",
"blobStorageUrl": "https://cloudconstructmedia.blob.core.windows.net/assets/albums/3f23e8a8-705e-4035-8ea5-ed9a528cc355/cloud-logo.png",
"isPublic": true
}

The Album Document :

{
"id": "53ab666f-d537-4ceb-a6dd-72cd76d4e525",
"_rid": "xdhrAPJZGQAFAAAAAAAAAA==",
"_self": "dbs/xdhrAA==/colls/xdhrAPJZGQA=/docs/xdhrAPJZGQAFAAAAAAAAAA==/",
"_ts": 1451928594,
"_etag": "\"00001500-0000-0000-0000-568aac120000\"",
"assetType": "Album",
"name": "My First Album",
"desc": "Test album",
"isPublic": true
}

The image is linked to the document via the albumId property as we expected. Awesome!! This alone is pretty cool and very useful because we already can support a much faster way to read this type of data using DocumentDB. Read operations in an embedded database are much faster and for this type of data model (which will expand with other properties such as tags, comments, geo-location, etc) we will want to pull it all back with an image without joining against tons of tables. This will be a great solution for a media library, but how could we make it even better? How about Azure Search...

Related DocumentDB Links :

https://azure.microsoft.com/en-us/documentation/articles/documentdb-modeling-data/

https://azure.microsoft.com/en-us/documentation/articles/documentdb-faq/

https://azure.microsoft.com/en-us/documentation/articles/documentdb-resources/

Step 4 - Azure Search with DocumentDB Setup

Azure Search on top of DocumentDB is a great solution if you want to offer excellent search functionality to your users. Within a photo library, users will want to query based on all sorts of criteria such as tags, comments, categories, and even geo-location. Azure Search has this functionality built right in by allowing you to define custom indexers with indexes and have them automatically kept up to date based on events occuring in your DocumentDB.

To connect your DocumentDB to Azure Search, simply go to the Azure portal and create a new Azure Search service. You may name your search service whatever you like, once it is setup by Azure you will need to do the following:

1. Map a Data Source by choosing Add Datasource from the portal.
2. Choose your DocumentDB from the dropdown.
3. Name the collection "Assets".
4. Set the Query to "SELECT * FROM c WHERE c.assetType= "Image" AND c._ts > @HighWaterMark ORDER BY c._ts". This is done to only return images in this search. You can also use this to flatten your JSON should it be nested.
5. Click OK

Once your datasource is connected to Azure Search you can now create an index to associate to the search.

1. Click Add Index in the Azure Search settings.
2. An index will be added, name it "imagesearch".
3. The fields should pre-populate for you, but if not then set them to the fields below. You will notice these property names match the name and type set in the DocumentDB. We will make all fields searchable besides id.

Once you save this new index, the indexer will run and if you only created 1 image document, index this document.

Now that your DocumentDB is indexed you can test out your search functionality by clicking the search explorer option in the portal. It is an option under the Azure Search settings at the top of the portal blade. Typing in "Company" which is a term in my image description returns the single document result.

Sweet!!

Step 5 - Connect ASP.NET MVC Application to Azure Search

Now we need to perform the final step which is to connect our ASP.NET MVC application to our Azure Search service so we can issue image searches from our listing page. Be sure to add the Microsoft.Azure.Search Nuget package to your project before continuing. You will need to check off the "include preview" checkbox in Nuget to get it as well.

1. Add the following application settings to your web.config file :

  <add key="SearchServiceName" value="MYSERVICENAME" />
 <add key="SearchServiceApiKey" value="MYKEY" />

2. Add a new class to your project called "ImageSearch.cs". You will see it references the Index named "imagesearch".

  public class ImageSearch
 {
 private static SearchServiceClient _searchClient;
 private static SearchIndexClient _indexClient;

 public static string errorMessage;

 static ImageSearch()
 {
 try
 {
 string searchServiceName = ConfigurationManager.AppSettings["SearchServiceName"];
 string apiKey = ConfigurationManager.AppSettings["SearchServiceApiKey"];

 // Create an HTTP reference to the catalog index
 _searchClient = new SearchServiceClient(searchServiceName, new SearchCredentials(apiKey));
 _indexClient = _searchClient.Indexes.GetClient("imagesearch");
 }
 catch (Exception e)
 {
 errorMessage = e.Message.ToString();
 }
 }

 public DocumentSearchResponse Search(string searchText)
 {
 // Execute search based on query string
 try
 {
 SearchParameters sp = new SearchParameters() { SearchMode = SearchMode.All };
 return _indexClient.Documents.Search(searchText, sp);
 }
 catch (Exception ex)
 {
 throw new Exception("Error querying index: {0}\r\n" + ex.Message.ToString());
 }

 return null;
 }
 }

3. Add the following private variable to your ImageController.cs file.

 private ImageSearch _imageSearch = new ImageSearch();

4. Add the following action method to your ImageController.cs file. This is our search endpoint for our application.

 public ActionResult Search(string q = "")
 {
 // If blank search, assume they want to search everything
 if (string.IsNullOrWhiteSpace(q))
 q = "*";

 var j = new JsonResult
 {
 JsonRequestBehavior = JsonRequestBehavior.AllowGet,
 Data = _imageSearch.Search(q)
 };

 return j;
 }

5. Replace the code in your Images/Index.cshtml file to match this:

@model IEnumerable<Image>

@{
 ViewBag.Title = "Index";
}

<h3>Index</h3>

<p>
 @Html.ActionLink("Create New", "Create", new { albumId = ViewBag.AlbumId })
</p>
<input type="search" name="q" id="q" autocomplete="off" size="100" />
<button onclick="Search();">Search</button>
<table id="searchResults" class="table">
 <thead>
 <tr>
 <th>
 @Html.DisplayNameFor(model => model.AlbumId)
 </th>
 <th>
 @Html.DisplayNameFor(model => model.Name)
 </th>
 <th>
 @Html.DisplayNameFor(model => model.Description)
 </th>
 <th>
 @Html.DisplayNameFor(model => model.BlobStorageUrl)
 </th>
 <th>
 @Html.DisplayNameFor(model => model.IsPublic)
 </th>
 <th></th>
 </tr>
 </thead>
 <tbody>
 @foreach (var item in Model)
 {
 <tr>
 <td>
 @Html.DisplayFor(modelItem => item.AlbumId)
 </td>
 <td>
 @Html.DisplayFor(modelItem => item.Name)
 </td>
 <td>
 @Html.DisplayFor(modelItem => item.Description)
 </td>
 <td>
 <img src="@item.BlobStorageUrl" alt="@item.Name" />
 </td>
 <td>
 @Html.DisplayFor(modelItem => item.IsPublic)
 </td>
 <td>
 @Html.ActionLink("Edit", "Edit", new { id = item.Id })
 </td>
 </tr>
 }
 </tbody>
</table>
<script src="http://ajax.aspnetcdn.com/ajax/jQuery/jquery-1.10.2.min.js"></script>
<script type="text/javascript">

 $(function () {
 // Execute search if user clicks enter
 $("#q").keyup(function (event) {
 if (event.keyCode == 13) {
 Search();
 }
 });
 });

 function Search() {
 // We will post to the MVC controller and parse the full results on the client side
 // You may wish to do additional pre-processing on the data before sending it back to the client
 var q = $("#q").val();

 $.post('/Albums/Images/Search',
 {
 q: q
 },
 function (data) {
 var searchResultsHTML = "<tr>";
 for (var i = 0; i < data.length; i++) {
 searchResultsHTML += "<td>" + data[i].Document.albumId + "</td>";
 searchResultsHTML += "<td>" + data[i].Document.name + "</td>";
 searchResultsHTML += "<td>" + data[i].Document.desc + "</td>";
 searchResultsHTML += "<td><img src=\"" + data[i].Document.blobStorageUrl + "\" alt=\"" + data[i].Document.name + "\" /></td>";
 searchResultsHTML += "<td>" + data[i].Document.isPublic + "</td>";
 searchResultsHTML += "<td><a href=\"/Albums/Images/Edit/" + data[i].Document.id + "\">Edit</a></td></tr>";
 }

 $("#searchResults").children("tbody").html(searchResultsHTML);

 });

 function parseJsonDate(jsonDateString) {
 if (jsonDateString != null)
 return new Date(parseInt(jsonDateString.replace('/Date(', '')));
 else
 return "";
 }
 };

</script>

6. Run your application. It should be pretty much all there at this point. You should see a textbox that allows for a search string to be entered and the results should be that single document you have indexed now.

Some Final Thoughts

Say you want to process these images as well after uploading. Creating thumbnails or special sizes for design elsewhere in your site. You have two good options in Azure : Blog Post on Azure Webjobs for Processing or use an Application for the Windows Azure Marketplace called Blitline. These will differ on cost due to the WebJob requiring you to create your own code, but there is probably some examples out there.

In Conclusion

I hope this blog post helped you walk through the basics of getting a simple media library up and running using Windows Azure technologies. The term simple does not do it justice though. This library is very scalable to accomodate millions of albums and images because it is built on technologies designed just for this type of work. Azure BLOB storage is a fast efficient way to store binary assets. It is also able to be set to various redundancy rates so you can be sure your data is secure and backed up should a disaster occur. DocumentDB is built to be be fast and efficient in storing embedded data that typically returned in a single database call. By grouping this common data properties in a single document, we can make our data reads so much faster. Lastly, Azure Search is an optimized search service with built in indexing and also has change tracking as well. We can easily just hit our search endpoint and get a type-ahead list of results, a complex list of results, or a large list of results very quickly.

I hope you enjoyed this article. It was a very fun project to take on and one I know will be helpful for our agency Cloud Construct's clients on in the near future.

Please contact Cloud Construct, a Boston-based digital agency, if you would like any help employing some of the best practices in your code and development process.

About the Author

Arra Derderian serves as the President and as a Lead Technical Architect for Cloud Construct, a digital development and web design agency based in Boston. As a founder at Cloud Construct, Arra is involved in all levels of the business from new project engagements, project planning, and development. Arra also serves as the founder of the Boston Orchard CMS User Group and is a member of the Windows Azure Insiders group. He graduated from Northeastern University School with a Bachelor of Science degree in Computer Science.