CosmosDB

Alan course notes

• Teach others this by saying, I read something about cosmosdb and would like to share.
• CosmosDB has various API like SQL API, table API, MongoDB API, Cassandra DB Api
• 
• Request_Unit decides the charges, you are not charged for memory or iops or cpu. You are also charged for space.
• Free account 400Request_Unit and 5 GB storage.
• While creating you need to give partition key and container ID.
• What system defined properties are added? 6 properties rid, self, etag, attachments, ts
• Github cloudxeus/Azure-dev
• nuget install Microsoft.Azure.Cosmos
• 
• In program.cs put connection string, database name, container name and partition key.
• When using Microsoft. Azure. Cosmos, the class Cosmos Client will be available which will take above parameters and give client.

Alan course transcript

now in this chapter i just want to give an introduction on to the CosmosDB service so this is a fully managed NoSQL database this is a completely managed service so here the underlying infrastructure is completely managed for you you don’t need to worry about the storage about the servers you can just start directly using the service it also provides fast access onto your data and it has features such as high availability and scalability now one of the unique features of azure cosmos tv is you can store different

types of data based on the api that you associate when you create something known as an azure cosmos db account for example if you choose the sql api then you can store json based documents and you can use sql like language to query for those documents you can use the table api if you want to store data in a table like format if you’re familiar with MongoDB and you want to have that in place in azure cosmos tv you can go and use that api if you want to host a Graph based database you can go ahead and use the

Gremlin api (Apache Gremlin) and if you want to use Cassandra DB that is also available when it comes on to this particular course we are going to be focusing on the sql api mainly and we just have a quick look at the table api as well now in this chapter let’s go ahead and create a resource based on the azure cosmos db service so over here in all resources let me go ahead and click on add and over here let me go ahead and search for the cosmos db service so i’ll go ahead and choose azure cosmos db i’ll go ahead and hit on create

over here i’ll go ahead and choose a Azure Resource Group an existing resource group over here i have to go ahead and give a unique account name so this name is not available now once you go ahead and enter the account name over here you need to go ahead and then choose what is the api so if you are going ahead and storing json based documents and you want to go ahead and have the ability to query those documents using a SQL queries like language you can go ahead and use the co sql api another option for storing json based

documents is to go ahead and choose mongodb so mongodb is only a database engine which is in place over here you have an implementation for mongodb using azure cosmos db if you want to go ahead and host a Cassandra DB on azure you can go ahead and choose the api accordingly if you want to go ahead and have a table like structure you can go ahead and use azure table and if you want to go ahead and have a graph based database you can go ahead and choose Apache Gremlin so since in the exam there is a lot of focus when it comes to the core sql api

let me go ahead and choose that i’ll go ahead and chooseprovision_throughput now when it comes to the capacity mode of the provision throughput you actually get charged based on the amount of storage and also the amount of something known hasRequest_Unit so in this particular mode you are not charged separately based on the underlying cpu that is used for hosting the cosmos db account you are not charged for the amount of memory separately or the amount of iops so everything is bundled has a metric known asRequest_Unit

and the request units is uniform no matter what api you choose please note that in addition to the request units you’re also charged for other aspects such as the amount of storage and remember the azure cosmos db service is a managed service you don’t need to go ahead and manage the underlying infrastructure everything will be managed for you now with azure cosmos db we have this facility of using thefree_tier when you get 400 request units and five gigs of storage for free in an account and this is fine if we just want to go ahead and

try out azure CosmosDB so i’ll go ahead and ensure to apply the free tier discount you can only go ahead and choose this for one account if you go out and create another account this discount will not apply i’ll go on next for global distribution i’ll leave everything as it is i’ll go on networking leave everything as it is so we can go ahead and connect from any network i won’t change anything in the backup policy nothing in the encryption let me go on to review and create and let’s go ahead and create this as your

cosmos db account now let’s come back once this is in place now once the azure cosmos tv account in place i’ll go ahead on to the resource and over here you will be directed onto the quick start menu when it comes to the features on the left hand side for the azure cosmos tv account now i’m going to go on to the data explorer and let me go ahead and just hide this so we have a better view of the data explorer so over here we have our sql api now the first thing we need to do is to go and actually create a new database

and then create a container for our items within the database itself so what i’ll do is that i’ll go ahead and directly create a container and over here we will be prompted to also go ahead and create a database as well so over here let me go ahead and give a name for the database over here when it comes to the provision data throughput you can go ahead and

Configure pay wall

choose the scaling as manual and over here you can go ahead and mention the amount of request units as 400 so as part of the free tier account you

as it is then i’ll go ahead and scroll down and over here let me go ahead and give thecontainer id now over here i am going to go ahead as an example and store customer information so i’ll go ahead and give the name of the container id has customer i’ll go ahead and scroll down and over here we have to go ahead and give something known as thepartition_key now i’ll explain partition key in a subsequent chapter over here the partition key is an

attribute of your items that will be used for distributing the items across multiple logical partitions so with azure cosmos db you might be storing a lot of data so you could be having millions of items in place in order to go ahead and help for the easy searching of items easy searching of data you can go ahead and ensure that your data is split across multiple logical partitions the search can then go into the required partition of choice instead of actually going through all of the items in the underlying container in the cosmos db

account so over here you have to go ahead and decide what attribute of your item is going to become thepartition_key now in the next chapter we are going to go ahead and see how to add these items so for the sake of simplicity i’m going to go ahead and choose customer id has my partition key so let me go ahead and choose that and then let me go ahead and hit on ok so now this is going to go ahead and do two operations it will go ahead and first create your database and then it will go ahead and create your container

and once this is done over here you can see your customer container within your app db database so let’s mark an end on to this chapter now in the last chapter we had gone ahead and create a customer container now let’s go ahead and add items onto this container so i can go on to items and over here i can go ahead and create a new item so your item is basically in json format and over here let’s go ahead and start adding our items so over here let me go ahead and add the first item which basically has two

properties so over here we have the customer id and the customer name and remember we have gone ahead and mentioned the customer id has a partition key so let me go ahead and click on save now once this is done you can see the item over here now in addition to the properties that you mentioned for the item there are also some system defined properties that have also been attached as part of the item over here one of the main properties is the id property of this item so now the id along with the partition key now helps to uniquely identify

your item within the azure cosmos db container the id property itself only helps to identify this item uniquely within a particular partition that’s it it is a combination of this both that actually help to uniquely identify the item within the entire container now i can go ahead and add an additional item now over here in addition to the customer id and the customer name i’m also going ahead and adding an array of orders so let’s say that this customer has gone ahead and placed an order to buy the

course az104 so i can go ahead and basically add that onto my container over here let me go ahead and hit on save so now we also have the orders array as well this is again an array of json based objects let me go ahead and add a new item and over here let me go ahead and now add another item again we have an array but this user or this customer has gone ahead and bought two courses so let me go ahead and replace it over here and click on save if you want to go ahead and make a change over here itself onto any item

you can go ahead and make the change and then you can go ahead and hit on update you can also go ahead and delete the item as well for now let me go ahead and discard the update right so over here now we have three items in place and now you can go ahead and execute your sql like queries against this data so over here you can go ahead and choose the option of a new sql query over here by default c (c for container) is the alias which has been given on to your customer container you can go ahead and hit on execute query and over here you’ll get all of the

items that are in your container you can then also go ahead and also get only the orders information so for that this is an example so if you want to go ahead and access that array from within your json objects itself you can go ahead and issue this sort of statement so let me go ahead and replace it over here and execute the query so over here you can see you’re getting only the orders information and if you want to go ahead and see the orders where in the course is equal to az1204 let me go ahead and execute this query

so now you’re only getting that information so in this chapter i want to show you how you can work withcontainer_items from within your container now before we actually go into working with data in azure cosmos tb so we’ve gone ahead and create an azure cosmos tb that’s based on the sql api so when you go ahead and start adding items in azure cosmos tv container that is based on the sql api you are going to be adding json based documents so for those students who are not familiar with json so i just want to go

ahead and give some points when it comes to json itself so json is nothing but javascript object notation so json was originally meant to have a much more easier way a much more simplistic way of representing data it also provide a means and easy means of exchange of data between software systems so if one system wanted to go ahead and talk or send a message on to another system they could go in and construct a json document with that information that json document could then be sent from one system on to the other

14:26

and since the json document was a very simple representation of the underlying data it became very easy for the underlying software systems to understand the data within the json document so all the data within a json document is represented has name value pairs so over here i’m giving a very simplistic example of a json document so over here we just have one item in the document the item contains two name value pairs so over here we have the customer id which has a value of one and the customer name which has a value

15:10

of user a over here the data within an item is separate by a comma you can then use curly braces to go ahead and hold objects so over here we have one object in place right so one object in place over here and you could also go ahead and make use of square brackets to go ahead and hold arrays so over here just trying to give an introduction when it comes to javascript object notation again don’t be so overwhelmed with the concept of json it’s just a very simplistic way when it comes to representing data

15:53

now in this chapter i just want to quickly go through the data types which is available in json and just give an example of some json items so json is supposed to be a very simple representation of data so you don’t have many complex data types but you still have data types in place that help json the engine basically understand how to represent the data so for example you have string data types you have number you have an object you have array and you have boolean you also have null as well now let me go ahead and give an example

16:33

of some type of json items now over here i’m showing you a document which has two json items or two json objects and each object also has an array so over here we have the first data value pair in the item that’s a customer id so over here the data type is basically a number then you have a string and then over here you have a representation of a boolean value so over here you have three data value pairs and you also have a fourth one but this time in the fourth data value pair it’s basically an array

17:19

of string values so over here we are trying to go ahead and represent customer information and those customers who have gone ahead and purchased a set of courses let’s say from an online provider so over here this is how you would actually go ahead and represent arrays in json now next i’m showing you another json document again this has two items so again we have our customer id the customer name and the boolean value and we also have an array of values as well but now within this object itself we have another

18:04

name value pair and this time this name basically is another object itself so if you want to go ahead and represent the details of a customer wherein each customer has the city and the mobile number you can go ahead and represent it has another object within the main object itself so over here this how you can actually go ahead and embed objects when it comes to json so again when you go ahead and actually just quickly compare the different ways you can actually store data so for the customers you could go ahead and store

18:44

the data in a simple comma separate file so this could be a csv file wherein you have the customer id customer name and all of the information so over here you can see that for each customer id because you want to go ahead and let’s say store the courses information you have duplication of data over here so when you look at the json representation we only have two items in place but because we can go ahead and store arrays over here we have less duplication of data if we want to go ahead and represent the same thing has a

19:27

csv file this is one way of representing the file now please know that you could also go ahead and add all the courses in one line over here but then when it comes to actually processing the file you need to do a little bit extra to go ahead and split those data values so if you want a simple representation of the data when it comes to csv this how you’d go ahead and represent the data and if you want to go ahead and represent the data has sql tables in a sql database you might have one customer table which could have the information

20:03

for the customer you might have another table for storing the courses which the customer has bought and you would go ahead and basically have a relationship basically foreign keys that map the customer table onto the course table right so i just thought that before we go ahead and play around with data in an azure cosmos db account for those students let’s have kind of a overview of javascript object notation or json now in the next set of chapters i want to go ahead and show you how you can work with azure cosmos tv

20:47

from dotnet programs so we are going to go ahead and first create a database and a container from a dotnet program so for that the first thing i’m going to do is to go ahead and delete the existing container that i have so i can go on to the context menu for the container and i can go ahead and hit on delete container and over here you just need to go ahead and give the container name and hit on ok once this is done let me go on to the database and delete the database let me go ahead and hit on ok then i’ll go ahead and just close this

21:30

query window i don’t require it anymore so now let’s go on to visual studio i’ll go ahead and create a new project so again we’ll use console based applications so i’ll go ahead and choose console application c-sharp go on to next so over here let me go ahead and give the name as cosmos db go on to next dotnet 3.

21:55

1 go ahead and hit on create now the first thing we need to do is to go ahead and use the proper nuget packages to work with as your cosmos tb so over here the first thing i am going to do is to go on to tools i’ll go on to new get package manager i’ll go on to manage nuget packages for the solution and let’s go ahead and search for cosmos dbe so i’ll go on to the browse section i’ll go ahead and search for cosmos db and over here i’m going to go ahead and choose microsoft.

22:29

azure i’ll go ahead and hit uninstall i’ll go ahead and hit on ok now once you have the packages in place i’ll go back onto my program and the first thing i want to do is to go ahead and define some properties so over here these are the properties i am putting into place so first is the name of database which i want to create what is the name of the container i want to create and what is the partition key for the container over here we have to go ahead and use the connection string which will allow this program to authorize itself to work

23:07

with azure cosmos db account so we have to go on to azure for our azure cosmos db account we have to go on to the key section over here let me go ahead and just hide this and over here in terms of the read write key so this will allow a program to go ahead and read and write data onto the azure cosmos db account so over here we can go ahead and take either the primary connection string or the secondary connection string so i’ll go ahead and take the primary connection string i’ll go on to my program and let me go

23:44

ahead and replace it over here now in order to start working actually with azure cosmos tv we need to go ahead and make use of the cosmos client class so i’ll go ahead and first ensure to use the namespace using microsoft.azure.com over here i’ll go ahead and choose the cosmos client option and over here we have to go ahead and give our connection string so let me go ahead and get the connection string over here now next we can go ahead and use that same client to now go ahead and create a database by using the create database

24:30

async method over here we have to go ahead and give a database name since this is an async method i’ll go ahead and ensure that the method goes on to completion only then we go on to the next statement now once our database is screened we can go ahead and get a reference on to our database so this database class is actually pertinent onto an azure cosmos db database so let’s go ahead and get first a handle on to our database which we have basically discrete in the last step so let me go ahead and do a get

25:15

database over here and once we have the database in place we can go ahead and execute the method of create container async and over here we can go ahead and give our container name and again get awata dot get result now over here i am getting a error because along with the container name we also have to mention what is a partition key so these two are very important because each container needs to have a partition key in place over here i’ll just say database and container have been created right so now let’s go ahead and run our

26:10

program right so this is in place over here let me now go on to the data explorer let me go ahead and hide this over here now i can see my app db database and my course container currently i don’t have any items in place right so in this chapter i want to show you how you can create a database and a container using dotnet code now in the last chapter we had gone ahead and seen how to create a database and a container in the azure cosmos db account now let’s go ahead and see in this chapter how do we add an item onto a

27:10

container using dotnet now for this we don’t need the partition key so this is required when we want to go ahead and create the container we do need to mention the partition key when creating the item but over here we have to go ahead and mention what will be the property in our class that will become the partition key over here i don’t need the statements for creating a database and i don’t need the statement for getting the database let me go ahead and also remove this statement as well so the first thing i am going to do is

27:42

to go ahead and create first a dotnet class to go ahead and represent our data so i’ll right click on my project i’ll go ahead and click on add and let me go ahead and hit on class so i’ll go ahead and create a simple course class so over here let me go ahead and add the attributes so first is the id so i said that every item needs to have an id property so i’m going it and having it over here and then over here since we also mentioned the course id has basically a partition key let’s go ahead and also have a property

28:21

for that as well so in your definition of data you could be having a separate id for the course information itself and you might have a separate id for the course itself so i have these properties in place now let me go on to my main program so first thing let me go ahead and create a new course object so over here let me go ahead and first mention the id has one then i’ll go ahead and mention the course id as well then i’ll go ahead and mention the course name and i’ll go ahead and mention a rating

29:20

right so i have my course object in place now i want to go ahead and add this as an item onto my container so the first thing i’ll do is that i’ll go ahead and use my cosmos client object to go ahead and get a reference on to my container so i can go ahead and use the method of get container and over here i need to first go ahead and mention what is the database id and the container id so i’ll go ahead and give my database name and my container name and this can be referenced now by an object of the class type of

30:01

container so container again is a class type which is you know specific for azure cosmos tv and now i can go ahead and use that container and i can go ahead and use the method of create item async so now over here i need to go ahead first of all and mention what is the type of item i want to create so over here i want to go ahead and create a course item and then it’s asking me for what is the item i need to create and what will be the partition key so over here in the item i want to go ahead and create my course and what is

30:45

the partition key so i have to go ahead and create a new partition key and my partition key is basically my course id now since this is a create item async again let me go ahead and do a get awaiter dot get result and this is done let me go ahead and just make the change over here right so we have everything in place now let me go ahead and run the program so it’s saying the item is created let’s go ahead and confirm so over here let me go ahead and hit on refresh and over here i can see my item in place so my course id is a partition key and i

31:46

have my id over here as well now in the last chapter we had seen how we could add an item onto a container from our dotnet code now i want to go ahead and show you how you can do a bulk insert of multiple items at a time so firstly what i’ll do is that for my existing item in my container let me go ahead and delete this item so i don’t have any items in this container and now what i’ll do is that insert is creating one course over here let me go ahead and create actually a list of courses so over here let me go ahead and ensure

32:31

that i use the namespace of system.collections.generic over here let me go ahead and mention this has the course id let me go ahead and mention a different id so i’ll go ahead and do this for all of the items or the courses that i want to go ahead and create right so everything is in place so now i want to go ahead and add the list of courses all at one time now when you go ahead and actually create the cosmos client connection over here you can actually go ahead and mention something known as client options so by default it is null

33:15

so in the client options there’s something known as a bulk insert that you can actually go ahead and mention and when you go ahead and mention this then the cosmos client will go ahead and use a much more efficient way to go and add multiple items at a time so i can go ahead and choose to create a new cosmos client options and over here there is a property of allow bulk execution i’ll go ahead and enable this or mark this as true now in order to go ahead and add all of these items i have to go ahead and

33:58

create now a list of tasks over here task new list of tasks i need to go ahead and use the system.threading.task namespace right so that is in place and now in this list i need to go ahead and add all of my create item async calls so over here let me go ahead and just take it till here let me add it over here so container.

34:41

createasync the type is course and over here i need to go ahead and basically give my list so let me go ahead and add now for each loop so for each course in my list so my list is basically over here my list of courses let’s go ahead and basically add our course over here the new partial key is the course id let me go ahead and complete the statement and then i don’t need this anymore i’ve gone ahead and added all my tasks i want to ensure that i execute all of my tasks so i’ll go ahead and use the task.

35:33

when all i’ll go ahead and add my task over here dot get awaiter dot get result so that it’ll go ahead and execute my task one over here let me go ahead and see bulk insert completed right so we have everything in place let me go ahead and run the program right so this is done let me go ahead and hit on refresh and now over here you can see all of the items in place so if you go on to any item here you can see all of the information so in this chapter i just want to go through the bulk insert feature which is

36:35

available as part of the net sdk when working with azure cosmos tb now in the previous chapter we had gone ahead and seen how to add a list of courses has items now in this chapter let’s go ahead and see how to read a particular item from an azure cosmos db container so over here let me go ahead and delete all of these statements so i don’t need the tasks also anymore so i just need to go ahead and ensure that i have a handle onto the client the first thing i’ll do is to go ahead and define my query so what do i want to

37:21

go ahead and get so over here the statement i want to execute is select star from c that’s our course table where the course id is equal to this value so over here for this value the course name is az303 now next we need to go ahead and create something known as an object of the query definition class so this will go ahead and hold basically our query so i’ll go ahead and say new query definition and in this i’ll go ahead and specify my query then i have to create something known as an object of the feed iterator class to

38:05

go ahead and ensure that we can i trade through the items as objects of our course class so over here let me go ahead and just give a name and then over here i need to go ahead and use my container dot get item query iterator and over here again my course class and then i need to go ahead and mention my query definition now once you have the feed iterator in place now we can go ahead and i trade through the items so we have to go ahead and use our iterator and we have to see whether it has any more results we’ll put this in a loop

38:48

so this is a property and over here we get something known as a feed response of the type course is equal to our itrata dot read next async so over here it will go ahead and read the next item from the azure cosmos tv container let me ensure that this statement also executes since this is based on threading a sync right showing you the first statement and then for each goes in our response so over here let me go ahead and get our response i can then go ahead and write our course id so over here let me go ahead and

39:42

mention has the course id if you want you can also get the id as well right and over here we can go ahead and remove this statement so over here just as a summary we are creating our query definition then we have to go ahead and create an iterator that will go ahead and i trade through the items in our collection and then for each response that we get we can go ahead and iterate through our courses so let me go ahead and run this particular program so over here you can see we are getting the course has per our query wherein we

40:38

are getting the course where the course id is equal to course 0002 right so in this chapter i want to show you how you can actually get an item from a container in azure cosmos dbe now in this chapter i want to show you how you can update an item in azure cosmos db container now earlier on we had gone ahead and seen how to read data from a container based on a query definition now remember for a particular partition you might have multiple items in place and that’s why you can actually go ahead and use the feed iterator to go ahead

41:25

and iterate through all of the items in our case we only had one item where this was the course id so when updating an item i’m going to go ahead and show you another way in which we can go ahead and get an item in azure cosmos db over there we just want to go ahead and get a single item and basically update the properties of that item so over here let me go ahead and remove all this so this is not required so over here let me go ahead and first mention what is the id of the item we want to get and what is the partition key

42:10

of that particular item so let’s say we want to go ahead and update this particular item where the id is 2 and this is basically the partition key which is our course id so i’ll go ahead and replace it over here so now over here has part of our container object there is a method known as read item sync over here we have to go ahead and mention the type of item we are trying to read and over here we have to go ahead and give what is the id and what is the partition key so over here let me go ahead and give

42:50

the id and i have to go ahead and create a new partition key over here i’ll go ahead and do the get awaiter dot get result now this particular method returns an item response so over here we have to go ahead and create now an item response object of the type course right so we’ve got the item over here now if you just want to get the details of the course itself so i’ll go ahead and first create a new course object and over here i can go ahead and use now my response object dot resource so this property will go ahead and return the

43:44

course itself and then we can go ahead and make a change onto the code so let me go ahead and change the rating so currently the rating is 4.6 so i’ll go ahead and change the rating on to 4.8 and then i can go ahead and use my container object and i can go ahead and choose the method of replace item async over here i need to go ahead and first mention what is the class type and then i have to go ahead and mention what is my item what is my id and what is the partition key so my item is this course and over here

44:23

i have the id and my partition key so let’s go ahead and first mention my item then the id and then the new partition key and again get awata dot get result right just showing you the full statement and then over here just saying that the item has been updated right so let’s go ahead and now run this program now once this is done so let me go ahead and choose another item and then go back on to the item with the id is equal to and here you can

45:27

see the rating has been changed now in this chapter let’s see how to go ahead and delete an item and this is quite simple i don’t need all of the item response etc so over here in the container object there is something known as the method of delete item sync again of the type course and over here we just need to go ahead and mention our id and our partition key that’s it so i’ll say item has been deleted so let’s go ahead and run this program right so this is done now over here if i go ahead and hit on

46:28

refresh so remember we have this course over here with the id of two so now you can see you don’t have that course in place so very easy to go ahead and delete an item right so with that we have gone ahead and complete our crud operations when it comes to working with azure cosmos tb please note that there are a lot of other methods as well when it comes to working with data with azure cosmos db now in this chapter i want to go through the change feed feature which is available with azure cosmos tb so with

47:10

the change feed feature if you go ahead and make a change onto items in an azure cosmos tv account the changes of that fee can be fed onto a destination that destination could be your application or it could also be an azure function so the azure function service already has the ability to go ahead and take the changes which are detected within items in an azure cosmos db container so we are going to go ahead and look at the change feature which is available as part of azure functions so over here in visual studio let me go

47:49

ahead and create a new project in terms of the templates i’ll go ahead and choose azure functions i’ll go on to next over here let me go ahead and just give a name and let me go ahead and hit on create now over here i’m going to go ahead and use the cosmos db trigger so this cosmos db trigger will actually be that change feed feature so whenever a change is detected now in the items in the container it will go ahead and trigger this as your function now over here we have to go ahead and give a name for the connection string

48:34

setting name not the connection string itself this setting that will be used to go ahead and store the connection string so over here let me go ahead and give a name over here we have to go ahead and give our database name and what is the name of our collection or container which is course so let me go ahead and hit on create so now over here there’ll be some default template code in place that will actually go ahead and basically look for changes in the container or the collection of course within our app db database

49:18

this is now based on the cosmos db trigger let me go ahead and just give a name over here i’ll change the name of the function as well now the first thing i need to do is to go ahead and specify our cosmos db connection string so for that let me go ahead and copy this i have to go onto the local settings.

49:57

json file and over here i can go ahead and add that setting so over here i need to go ahead and add the connection string so in azure let me go ahead on to the keys let me go ahead and take either the primary or the secondary connection string so i’ll go ahead and take the primary connection string and let me go ahead and replace it over here let me go back on to detect changes now over here you will notice the use of something known has a lease collection name so in order to go ahead and basically record the changes over here a new container or a

50:46

collection is going to be created in your app db database the name of the collection will be leases now over here i’ll also go ahead and mention to create the lease collection if it does not exist so i don’t have the lease collection currently in our app db database so let me go ahead and ensure that it creates this lease collection now when i’m reading information about the items or the documents that get changed it will be mapped onto our course class which is basically mapped on to the item structure that we have in our container

51:28

or collection so over here let me go ahead and create a new class so add a new class and over here let me go ahead and mention the course class and over here let me go ahead and mention the properties so now over here if multiple documents have been detected that have been changed over here it will go ahead and look at the count so the count is greater than zero it will basically go into this loop now i want to go ahead and i trade through all of the items that get changed so for each course item that actually gets changed i want to go ahead and

52:14

write all of the information so over here let me go ahead and change this has the course id so over here get the property value of the string since this is a specific property value right so that’s done let me go ahead and also log the information about the id itself so that will already be available the id property right so now that we have this in place let me go ahead and now run the program right so we have the program running over here

53:17

now let me go on to my data explorer i’ll go on to my course items you can see now you also have a leases collection in place let me go on to any item and let me go ahead and make a change i’ll go ahead and hit on update and over here now you can see the information in place so sometimes there can be a requirement to go ahead and look for changes that have been made on to your data in azure cosmos tv and you might want to go ahead and process those changes so instead of actually pulling those changes instead

54:00

of actually pulling for those changes over here you’re automatically getting the changes via the change feed feature now in this chapter let’s go ahead and see how to create first a simple stored procedure so over here i can go ahead and create a new stored procedure so over here everything is basically in javascript so let me go ahead and remove this so over here i’m going ahead and giving a name for the stored procedure and over here i’m giving a function so over here in the function i’m just

54:44

getting the context of the request i’m going ahead and getting a handle onto the response and i’m resetting the response body has executing a stored procedure a very simple stored procedure over here let me go ahead and click on save now once this is saved so i’m going to go on to the program wherein we interact with our azure cosmos db container now over here in the container object we have something known has scripts and over here we can actually go ahead and execute a stored procedure a sink over here we have to go ahead and now

55:29

mention what is basically the return type so it’s going to go ahead and return a string so if you go on to the function over here in the body it is going ahead and returning a string and then next i need to go ahead and specify what is the name of my stored procedure and what is the partition key now currently we don’t have any partition key because our function our stored procedure is not doing anything it’s only going at and returning a response so over here let me go ahead and first mention what is the name of my stored

56:13

procedure so over here this is the name of my stored procedure and then over here let me go ahead and just create an empty partition key so i’ll go ahead and mention a string dot empty right and over here let me go ahead and do a get awata dot get result and since this is returning a string let me go ahead and get the output i also need to go ahead and mention some options over here so these options are specific when executing a stored procedure for now since i don’t have any options i’ll go ahead and specify it as

56:55

null and then i can go ahead and write the output onto the console so let me go ahead and run this program right so we’re getting the output over here executing a stored procedure now let’s move on to the next chapter wherein we’ll look at another example on using a stored procedure now in this chapter i want to go through another example on using stored procedures so first of all in my azure cosmos db account let me go ahead and create now a new stored procedure and over here i’m going ahead and adding

57:52

code that is also available in the microsoft documentation so what this function does is that it goes ahead and adds a new item onto our collection or container so over here if you go ahead and scroll down it goes ahead and uses the method of creating a document now over here this particular function expects items in place other items will represent the data for the items that need to be created in our container so over here once this is actually done it will go ahead and set what is the count in terms of the number of items

58:35

that were added now over here let me go ahead and add a name and let me go ahead and hit on save now once this is done let me go on to my program now over here i am going ahead and creating basically a dynamic array of items you can go ahead and add multiple items over here of what i want to add into my collection or container in my azure cosmos db account now over here in my scripts again i have a string has the return type over here i need to go ahead and change what is the name of our stored procedure so it’s add item

59:32

so over here you can see that it is add item now over here we have to go ahead and define now our partition key so our partition key is basically the course id let me go ahead and copy this i’ll go ahead and replace it over here now i also need to go ahead and mention what are the items i want to go ahead and pass on to my stored procedure now over here before i run the program just making sure that this is course id because this is the name of my partition key and let me go ahead and now run the program so i can see that the number of items

1:00:21

that were added is one now if i go on to my items over here if i go ahead and hit on refresh so now i can see the new item in place now in this chapter i want to go ahead and show you an example on how you can work with triggers in azure cosmos tb so over here let me go ahead and create a new trigger so the trigger type could be pre opposed so if you want something to be done before an action actually happens on an item on the azure cosmos db database site you can go ahead and choose a pre-trigger if you want something to be done after

1:01:10

an operation has been complete and as your cosmos db you can go ahead and choose a post trigger now i’ll go ahead and choose a pre-trigger and over here i can go and choose all of the operations or you could also go ahead and select a very specific operation now over here i’m going to go ahead and add some code so what i’m trying to do is that i want to go ahead and get what is the body of the request so over here we are going to go ahead and use dotnet to create an item in our azure cosmos db account

1:01:43

then i want to get the details of the item and if the item does not have a timestamp property then i will go ahead and create a new timestamp property and i’ll go ahead and add it onto the item so now the item when it is added onto the container or the collection in the azure cosmos db database it would also go ahead and have this timestamp property so over here let me go ahead and give a name and let me go ahead and hit on save so now it’s going in and creating the trigger right so this is done so if you go on to

1:02:29

the items if you go on to any items you can see that in the past when we are going at and adding items there is no timestamp property over here there is a ts property but over here just to showcase how you can add an additional property i’ll go ahead and add a property with the name of timestamp now over here in my.

1:02:52

net code let me go ahead and first create a new course object so over here again we’re getting a handle onto the container so over here just mentioning what are the properties of my object so let me go ahead and just change this on to eight and let me go ahead and put an id of six over here let me go ahead and just change the name right so once this is done so we’ve gone ahead and create our new course now if i go on to my container and if i go ahead and now create the new item so this should be the same way over here let me go ahead and choose

1:03:44

course now over here let me go ahead and mention what is my item here i don’t need to mention the partition key and now i’m going to go ahead and add now a new item request options object and over here i can go ahead and mention what is the name of my trigger so let me go ahead and just confirm on what is the name of my trigger so if i go on to my triggers let me go ahead and copy this i’ll place it over here so this is for creating the item and then let me go ahead and ensure that the result is in place

1:04:32

and then i’ll just write a simple message that says that the item is created over here let me also go ahead and just confirm when it comes to my connection string so over here i don’t need the allow bulk execution since i’m only adding one and i’m basically using now a trigger as well now let me go ahead and hit on run and now i can see the item is created let me go ahead and close this going back on to azir if i go on to my items let me go ahead and hit on refresh so i can see over here the course where

1:05:22

the id is equal to 6 and over here now i can see the additional timestamp property so this has been done via our trigger so in this chapter i want to kind of show you how you can actually work with the triggers that is available for us your cosmos dbe hi and welcome back now in this chapter i won’t talk about embedding your data when it comes to items so again we are referring to azure cosmos db when it comes to the sql api now in a traditional relation database so let’s say you’re storing data when it

1:06:04

comes to an e-commerce system and let’s say you want to go ahead and store orders that have been placed by a particular customer so normally in a relational data model you might have one table for your customer you might have one table for your orders another table for your product and then you go ahead and basically have join so you’ll have primary keys you’ll have foreign keys you’ll ensure that there are the required relationships between these tables now when it comes to azure cosmos db there is no concept of having a

1:06:33

relationship with another container or within containers there’s no such thing as the foreign key and the primary key so what is the other way in which you can actually go ahead and relate related items so one way is to go ahead and embed data in a particular item that we are seeing over here so normally in a traditional relation database you’ll actually go ahead and normalize your data this is so that you don’t have to replicate the same data over and over again but when it comes to the nosql design

1:07:10

practice here actually what you do is that you go ahead and denormalize the data so over here i’m giving an example of an item wherein we have a particular customer and within the item itself we are going ahead and embedding the data about the order that is being placed by the customer over here we can actually have an array of orders so this itself is one particular object so the order id the product id and the quantity so we can have multiple orders that are linked on to this customer now over here i’m just giving an example

1:07:55

so if you don’t have many orders per customer this is one way of doing it if you have too many orders per customer then you have to go ahead and you know do another design practice wherein you actually reference data so i just want to give you a very simple example on how you can actually embed data within an item so normally the recommendation to embed data is when there is contained relationships between the entities so over here there is a relationship between the customer entity and the order entity when there is a one to few relationship

1:08:30

between the entities itself now this might not be the perfect example for this as i said you might have multiple orders for a particular customer and if you have a limit to these orders then it’s fine to go ahead and embed the data but when you have a large number of orders this might not be the most you know logical design when it comes to embedding data but i said i just want to give you a simple use case scenario on how you can embed data also consider embedding data when the embedded data itself changes

1:09:04

infrequently that means you don’t have to go ahead and change this particular order so normally when you actually have an order in place it’s very it’s not frequent to go ahead and change the order itself yes a customer might change the order but it’s not that all the orders are going to change so try to embed data also that does not change frequently because then you have to go ahead and change the embedded data probably in many documents now why is this the case is because i said you can actually duplicate data in

1:09:35

nosql here you’re denormalizing the data so the same order data might be in another item in another document let’s say in the same container and if you were to go ahead and let’s say have to change any one of the properties of the order we have to go ahead and change the properties of the order in the other document as well so try to ensure that the embedded data also does not change frequently also when the embedded data is queried frequently along with the primary data then also it’s a good case to go ahead

1:10:09

and embed the data hi and welcome back now in this chapter i want to go through an example on how you can embed data in items when adding them onto azure cosmos db from a net program so i’m making use of a program an earlier program we had seen where we could go ahead and insert an item into an azure cosmos db collection or container over here this was based on the object for my course class so over here i have the details about my course class so over here i have the details about the object i was creating and then i was going in and using the

1:10:59

create item async method to go ahead and basically add this object has an item and over here i was mentioning the partition key now over here if i go onto the items in that container in that collection so currently i don’t have any items in place now let’s say that i also want to go ahead and ensure that in this item along with the course we also have the order information as well so the first thing i am going to do is to go ahead and create a new class in my project so over here let me go ahead and create

1:11:40

a class with the name of order then i’ll go ahead and just add simple properties for this order so the order id and the price now i’ll go on to my course class and over here let me go ahead and add another property this will basically be a list of orders so over here i’m embedding a list of orders within my course class now if i go on to my main program and now over here in my main program i can now go ahead and create a new list of orders so over here let me go ahead and ensure to use systems.collections.generic

1:12:42

over here let me go ahead and create a new order so i’ll go ahead and basically create two orders and let me go ahead and ensure that i assign it onto my orders property of my course class right i have this in place so everything is done let me go ahead and now run the program right the item is created now if i go ahead and hit on refresh so i can see my item in place and over here i can see my order information has an array so over here now i’m embedding basically json objects in an array within my main

1:13:38

json item itself hi and welcome back now in this chapter i want to go through composite indexes which is available for azure cosmos tv now for this i am going to go ahead create a new collection in my azure cosmos db account that is based on the sql api and then i’m going to go ahead and add these items onto the collection over here this is going to be my partition key so let me go ahead onto the data explorer let me go ahead and hide this so i already have a database in place appdb over here let me go ahead and create a

1:14:25

new container over here i’ll give the name of the container has course and over here i’ll go ahead and mention the partition key and let me go ahead and hit on ok now once i have the collection in place let me go on to the items over here let me go ahead and start adding the items one by one so i’ve gone ahead and added all of the items so over here you can see i have five items in place now if i go and create a new sql query and if i go ahead and try to order the items by two properties the course name

1:15:16

and the rating you’ll actually get an error and that’s because you need to have a composite index in place to go ahead and serve this query you need to have a composite index that is based on the course name and the rating so over here i’ll go on to scale and settings over here in the settings i’ll go on to the indexing policy and over here in the policy let me go ahead and append something onto the policy itself so over here after the excluded parts let me go ahead and add now composite indexes wherein i’m adding the composite index

1:16:07

of the course name and the rating in ascending order right let me go ahead and complete this and let me go ahead and hit on save now once this is done if i go on to my query if i go it and again execute this and over here you can see you’re getting the output as desired so in order to go ahead and ensure that you can order by multiple properties of your items you have to go ahead and add composite indexes now in this chapter i want to go to the time to live feature which is available in azure cosmos tv now over here you can go ahead and

1:16:49

specify a time to live for all items in a container or for just a specific item after the time to live setting is reached the item will be deleted automatically the item will be deleted using the leftover request units if there are no leftover request units then the deletion of the item will be delayed now you can go ahead and set the value of the time to live at the container or the item level so i’ll just go ahead and show you how to implement the time to live feature so over here in our container so i have

1:17:26

some items in place now let me go on to the item where the id is equal to three and over here i can go ahead and add another property that’s known as ttl or the time to live and over here i can go ahead and mention a value of 20 seconds and i can go ahead and hit on update now in order for this setting to actually take update there is one more thing that we need to do we have to go on to scale and settings for our container over here we have to go on to settings and over here we have to go ahead and set the time to lift property for the

1:18:05

container and we have to go ahead and then hit on save now once this is done i’ll go on to items and let’s wait for a duration of 20 seconds now after waiting for a duration of 20 seconds let me go ahead and hit on refresh and now i can see the item with the id of three is no longer present over here so you won’t be able to go ahead and make a successful update onto this item anymore now if you want to go ahead and specified at the container level again go on to scale and settings go on to settings and over here turn this on and

1:18:46

over here you can go ahead and mention the settings which will apply onto the entire container so let me go ahead and hit on save once this is done go on to items and let’s wait for 20 seconds now i waited for 20 seconds let me go ahead and hit on refresh and now i can see no items in my container so over here i just want to go ahead and show you how you can work with the time to live property hi and welcome back now in this chapter i want to go through some commands from the azure cli perspective this is when it comes to creating us

1:19:36

your cosmos db accounts the first command is to go ahead and create an azure cosmos db account over here this is the name of the account the resource group over here we are mentioning the default consistency level we are giving a location we are seeing in best u.s 2. over here we are not giving any sort of failover priority so let me go ahead take this command over here in azir cloud shell let me go ahead and clear the screen and let me go ahead and paste in this command if i just go on to new tab if i go on to my azure cosmos db accounts

1:20:25

secondly i have two azure cosmos db accounts in place they should go ahead and create another one so let’s wait till this is complete now over here i can see i have an output in place so in another tab if i again go on to my azure cosmos tv accounts so i can see my new account in place now the second command is going to go ahead and create another azure cosmos db account but over here i am going to go ahead and give now multiple locations so over here i’m giving the primary location as west us-2 another location as east us too

1:21:11

and over here i’m giving the second location a field or priority of one so let me go ahead and execute this command so i’ll just go ahead and clear the screen let me go ahead and paste it over here and hit on enter now let’s come back once this is complete so i can see the operation is complete now in the other tab if i go onto my azure cosmos tv accounts if i go ahead and hit on refresh so i can see my new account in place now let’s go ahead and create another azure cosmos tb account now over here

1:21:52

the difference is that i want to go ahead and base this on the table api so in the capabilities i’m going ahead and mention it has enabled table so let me go ahead and also copy this so over here let me go ahead and clear the screen over here let me go ahead and paste this so while this is being done just a quick note so in this account where we mentioned multiple regions over here now you can see different read locations right so let’s come back once this is complete now once this command has executed

1:22:32

successfully in another tab i can actually go on to the cosmos db account over now over here now you can see that this is based on the table api so in this chapter i just want to go through the important commands when it comes to azir cosmos db when it comes to the azir cli hi and welcome back now in this chapter let’s go ahead and create an azir cosmos db account that is based on the table api so over here let me go ahead and click on add over here let me go ahead and choose azure cosmos db i’ll go ahead and choose one of my

1:23:23

existing resource groups now over here in the api i’m going to go ahead and choose azure table over here let me go ahead and give a name for the account right so it’s not taken i can leave the location as it is over here i can go ahead and choose provision throughput let me go on to next for global distribution let me go and keep everything as it is nothing networking nothing in backup nothing in encryption let me go on review and create and let’s go ahead and create this as your cosmos db account let’s come back once we have the account

1:24:00

in place now once you have the azure cosmos tv account in place you can go ahead on to the resource over here you can actually go on to the data explorer so the experience is the same no matter what api that you have it’s only that the data that you actually create in the underlying account will be a little bit different based on the type of account that you create over here there is no concept of a database or a container or a collection over here you have to go ahead and create now a table over here you can go ahead and give a

1:24:34

name for the table over here you can go ahead and specify the throughput and you can go it and hit on okay once you have the table in place you can go on to entities and the concept is similar to the table service we have seen with azure storage accounts over here you can go ahead and add an entity onto the table and over here again you have a concept of the partition key and the row key so if your partition key is the city in which the customers stay you can go ahead and specify that has the row key if you want to go ahead

1:25:19

and specify the customer id and you can go ahead and add a property wherein you have the customer name and you can go ahead and specify it over here and you can go ahead and add the entity you can go ahead and add multiple entities and then you can go out and even query for the entity so the experience is the same has a table service with azure storage accounts now in this chapter i want to show you how you can add an entity onto the table in the azure cosmos db account so you can go ahead and take the exact same program

1:26:02

that is used for adding an entity on to the table service in azure storage accounts and use it to add an entity onto the table api in azure cosmos tv that’s why i’m not going through all of the operations in the table api for as your cosmos db because you’ve only gone out and seen it for the table api for azure storage accounts over here the only thing i am going to change is basically the connection string so if i go on to my azir cosmos db account over here let me go on to the connection string over here i can go ahead and take either

1:26:46

the primary or the secondary connection string so i’ll go ahead and take the primary connection string i’ll go on to my program and i’ll replace it over here so i have my customer table in place so over here if i go on to data explorer let me go ahead and hide this so over here i have my customer table in place if i just go over on to the entities let me go and just delete this entity right so i have no entities in place now if i go ahead and run this program so it has gone ahead and added the empty

1:27:42

over here if i go ahead and close the entities tab and go onto entities again so over here i can see my entity in place so the exact same program can be used to work with the table api for azure cosmos tv hi and welcome back now in this chapter i just want to go ahead and showcase a very simple program in which you can actually go ahead and take data from a json base file and then you can go ahead and add the data has items onto a collection in an as your cosmos db account so over here this is a console based application so i have a folder in place

1:28:31

and in that folder i have a json file so over here i have multiple json objects in place so over here i have information such as the correlation id the operation name the status event category etc now i want to go ahead and add this information onto a collection in asia cosmos tv so firstly i’ve gone ahead and create an activity class so this represents the data that i have basically in my json file so i have the correlation id the operation name the status the event category and the level the only additional property is the id

1:29:16

and this will be id for each item in the azure cosmos db collection if i go on to my main program over here so i have the connection string the database name and the name of my container over here i’m actually going in and using the operation name has the partition key so let me go ahead i have my azir cosmos deep account in place this is based on the sql api the first thing i’ll do is to go ahead and create a new container so over here let me go ahead and give the container name of activity and let me go ahead and scroll down

1:29:58

and put the partition key so i’ll go on to my activity class i’ll go ahead and take the operation name i’ll just copy it i’ll go ahead and place it over here has the partition key and let me go ahead and hit on ok right so if i go on to the activity if i go on to items i should not be having any items in place over here now going back on to my program class so let me go ahead and also ensure i have the proper connection string actually let me go on to the key section over here let me go ahead and take the

1:30:44

connection string let me go ahead and replace it over here just to ensure that i’m connecting on to the right as your cosmos db account now i’m going ahead and basically first reading my query.json file using the file stream class i’m going ahead and then reading it onto a stream reader and then i’m going in and reading onto a json text reader over here i am getting a handle onto my cosmos db account and on to my container and now for my json reader once i go ahead and find out that a new json object is going to be in place

1:31:29

so over here if i go on to query.json so basically this is one json object and i want one object to be added as an item onto the azure cosmos db account so as soon as a new json object is detected that means i want to go ahead and add this as an item onto my azure cosmos db account so going back onto my program then i will go ahead and load the previous item whatever i’ve read so far into a json based object then i’ll basically type cast it on to my activity class i’ll then go ahead and generate a unique

1:32:15

id for my activity object and then i’ll go ahead and add it onto my container over here i’m going in and mentioning the partition key for this program what’s very important is that for the query.json file so if you actually go on to the properties i’m ensuring to always copy this file onto the output directory so that it is available for my program now let me go ahead and run this program so now this will continuously go ahead and start adding items onto the azure cosmos db account if i go back now onto the data explorer

1:33:14

if i go on to my activity if i go on to items and now i can see the items are being added on to my container so this will keep on going on now i’m going to go ahead and keep my program running for around 10 minutes because i have a lot of data in that json file now has an optional chapter just for learning purposes i’m going to go ahead and show you how i generated that data file so normally when working when practicing along it’s always important to have data in place and i’ll just show you how i

1:33:53

generated that json based file but for now let’s wait for some time i want some items to be added on to my container now i’ve come back after some time so over here i can see my items are still being loaded over here if i go on to the query let me go ahead and do uh select count star from c let’s see if this works i think i need to go ahead and mention at least one property over here right so now it’s fine so it’s gone it and insert around 3000 items now if you go back onto your azure cosmos tv account if you go and scroll

1:34:38

down if you go on to the inside section so this will give you kind of an overview of what’s happening in terms of the monitoring of your azure cosmos tv account over here you can see the total number of requests if you go ahead and scroll down over here you can see the ru consumption the request units that you are consuming over here you can see the data that’s being used over here you can see the total request so everything now is in this format when you have some sort of a visualization of what’s happening

1:35:12

you can go ahead and look at just the throughput you can go out and look at what are just the requests so over here you can see the total that’s the sum of the request units right so in this chapter just want to kind of show you how we can go ahead and stream continuous data onto azure cosmos tb so in this chapter i mentioned before just has a learning activity i want to go ahead and show you how i generate that json based file so i went on to the azure monitor section over here if i go on to activity logs

1:35:57

so activity logs goes ahead and basically logs all the activities that are happening as part of your azure account so any sort of administrative operation that you perform whether it be creating a virtual machine whether it be creating a storage account whether it be listing the keys of the storage account everything will be logged over here so if you have a substantial amount of activity you can actually go ahead and use this information so over here i’m just sharing my knowledge on how i generate data or how

1:36:31

i use data in the learning process when it comes to learning as your services now over here i can go ahead and choose a time span so i can see that i want all the activities for let’s say the last month and hit on apply so that i get more activities over here then i can go ahead and download has a csv now i have gone ahead and used as your data factory that’s another service that’s an etl tool an extract transform and load tool to go ahead and convert that csv file onto a json based file now obviously you

1:37:15

can use any tool that is available that has the facility to go ahead and convert the csv file into a json-based file but normally since my purpose is to go ahead and learn about seo it’s always good to go and try out services in azure that you haven’t tried before so since i am familiar with azure data factory that’s why i’ve gone ahead and use that service to go ahead and convert my csc file into a json based file so while the download is happening let me go on to all resources and let me go ahead and create a

1:37:52

resource based on azure data factory so i’ll go ahead and choose azure data factory and let me go ahead and hit on create over here let me go ahead and choose a resource group over here let me go ahead and choose my region over here let me go ahead and give a name for my data factory i’ll go onto next for git configuration so i’ll configure git later i’ll go on to networking leave everything as it is i’ll go on to advanced i’ll go on tags i’ll go on review and create and let me go ahead and create this

1:38:32

instance of azure data factory now to create azure data factory there’s no cost involved but then in azure data factory we are going to go ahead and create a pipeline that is going to go ahead and convert that cc file into a json based file and there is a very minimal cost when it comes to performing that activity so over here i’ll go ahead on to the resource i’ll go on to the author and monitor section here you basically have a wizard where you can go ahead and start using as your data factory over here quickly is going on to the

1:39:12

pricing section for azure data factory the pricing starts from a dollar per thousand activity runs per month and we are only going to be performing one activity run now i can always see that the download is complete for my csv file now we’re going to go ahead and upload this csv file onto it as your storage account so azure data factory is going to go ahead and take this csc file from an azure storage account create a json base file in that same storage account so i’ve just gone ahead and copied that cc file onto a folder on my local

1:39:54

machine let me go ahead and close this so over here we are as your data factory now in another tab let me go on to storage accounts so i have a lot of storage accounts in place if you don’t have one you can go ahead and create a storage account let me go on to one of my storage accounts over here if i go on to containers we can go ahead and create a new container so i’ll go ahead and hit on create i’ll go on to the container and let me go ahead and upload that file from my local system so i’m going ahead and uploading that

1:40:37

query result file i’ll go ahead and hit on upload right just gone ahead and uploaded that file over here i am just opening the file so you have a lot of information in place so over here the first row is the header information about all of the activity logs and here you have all of the data now let me go on to the data factory let me go ahead and hit on copy data over here you can go ahead and give a name for the task i’ll go on to next to run it once now now over here we have to go ahead and specify the source and the destination

1:41:21

so over here i’ll go ahead and create a new connection for the source you can go ahead and connect onto a variety of sources in azure data factory i’m going to go ahead and choose azure blob storage and hit on continue i’ll leave everything has it is and over here let me go ahead and choose my subscription and let me go ahead and choose my storage account let me go ahead and hit on create so over here we’re trying to go ahead and link azure data factory onto our storage account i’ll go on to next for the data set

1:41:57

over here i can now go ahead and browse for my factory folder so i’ll go ahead and double click and choose my query result.csv file i’ll go on next now over here it will automatically go ahead and detect the format of that file over here we have the option of specifying the first row as the header so it’s going in and specifying that information and here we have all of the data the preview of the data basically now i’ll go on next now i want to go in my destination and choose my same storage account i want

1:42:36

the json file to be stored in that same storage account so i’ll go ahead and hit on next over here i’ll go ahead and browse for the same path so i’ll go ahead and choose the factory path so i’ll go ahead and choose the factory path i’ll go on to next now over here in the text format i’m going to go ahead and choose the json format over here in the file pattern i’ll go ahead and choose it has an array of objects i’ll go on to next now over here you can actually go ahead and copy all of the columns but what

1:43:14

i’ve done is that i’ve actually gone ahead and ensured that only some of the columns are copied so i’ve not copied the time the subscription the event initial buy is to make it much more simple so over here you can decide what is the data that you want to go ahead and copy you can go on to next leave everything as it is go on to next and now it will actually go ahead and create a pipeline that pipeline will be using underlying compute infrastructure to go ahead and perform the conversion process you can go ahead and hit on monitor over

1:43:49

here and over here you can see your pipeline in place over here currently it’s in progress but this should finish quite fast if i go and hit on refresh you can see it’s already done and now if you go on to your factory container if you go ahead and hit on refresh you can see your json file in place you can actually go on to the file you can go ahead and download the file so i’ve copied it on to my temp folder let me go ahead and just open up in notepad and over here you can see you have your array of json objects

so i said over here just as extra learning just want to show you what i’ve done to go ahead and generate that json based file

other

name Azure code samples Github
type link
action https://github.com/Azure-Samples/cosmos-dotnet-core-todo-app

name Basics intro
type link
action https://dev.to/rupeshtiwari/azure-cosmos-db-basics-1dhn

You may have only one container in the database. However, All the items in a container could be have different types and Schema so you may need more containers.

Partition_Key is about how Cosmos DB stores the physical multiple documents together based on the partition key you define that is common property in all documents. eg. IMO number

Database > Container > Documents

Request_Unit (RU) is the charge for the request.

Notes

Intro

• Q. What will be the consistency level of cosmosDB?
• There can be any number of partitions within a container, no limit to that.
• Q. How to decide partition key?
• Choose a value which don’t change.
• Copy data from old to new container if changing partition key.
• Request unit is a measure of CPU, Iops and memory.
• You can provision throughput at container level or db level??
• cosmosDB and SQL API data is stored as JSON documents.
• Q. How to add items to a container? What is covered in that?
• New items can be added from Azure portal as well.
• Three partition can be created for mobile, laptop and desktop.
• When creating a container we give the partition key.
• Partition key can be orderId
• SQL queries can be used like Select * from Orders WHERE orderId = ‘1’
• Create customer.json and order.json for the casestudy in course.

Dotnet integration

• Create database

Questions

• Q. What are the security concerns in CosmosDB, is there possibility ofSQL_injection ?
• Q. If the hierarchy is DB > Container > Document, then what is a collection?
• Q. How to createstored_procedure in CosmosDB?