Friday, December 18, 2015

Jenkins quick setup on @Nitrous

Want to quickly setup Jenkins on the cloud? Try this on http://www.nitrous.io. See a prior post about setting up Dashing.io on Nitrous.io for basics of how to get started on Nitrous.io if required and then come back here.

Advantage:

If you want to experiment with something new on Jenkins before you run it in a production environment, this will get you going quickly. If you mess up, just re-install in Nitrous.io and start over.

Step 1: 

Copy the installation commands from this Jenkins-ci page - https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins+on+Ubuntu to a shell script in the Nitrous.io IDE and run the shell script. You could just run them in the command line if you choose.



Step 2: 

View Jenkins in the browser by previewing port 8080 as seen in the screenshot



Jenkins will show up on a new browser window. It works! 





Monday, December 14, 2015

Learning iOS development on @MacinCloud

I have been wanting to learn and develop iOS apps or a while. The price of a good developer's Mac was a little unreasonably high and I am quite sure I am not ready to switch over 100% to a Mac anytime soon. I found MacinCloud.com and their pricing is very reasonable. I didn't want to own a Mac, I didn't want to spend a lot of cash, but I wanted to use a Mac for a few hours a day just to learn iOS development. MacinCloud.com just worked out fine.

I signed up for the $20 per month service to access a real Mac (not a Virtual Machine) for three hours a day. The device they gave me is a high end Mac Mini with 16GB of RAM and preloaded with all the popular app development tools and platforms out there, including XCode, PhoneGap and a lot of others that I am not even familiar with. Google drive and Dropbox are already installed and so is Chrome and Firefox. Save all your data in Google Drive or Dropbox, share with your team or move your content to your personal laptop. Too many options, but there is no admin access. I was fine with that because they had everything I needed pre-installed and ready to go.

I have options to login through Remote Desktop or login via browser. I didn't even try the browser option. There are other options for accessing from a iOS device, but c'mon why would anyone want to do that?

The customer service is great too. I was having issues with performance at night time. I sent an email to them, they responded by next day morning, asked me to run some speed tests and traceroutes. The next day, they moved me to a different server and it has been working great since then.

Brand new Mac Mini with 2.6GHz processor and 16B of RAM: $899

Yearly cost of the same device on MacinCloud: 20 x 12 = $240

It is a better deal than owning a Mac for what I want to do with it. I think I will stick with MacinCloud.com for now.

Saturday, November 28, 2015

@Jenkinsci analytics with @Elasticsearch and #Kibana

I have been doing a lot of work with presenting Jenkins data in dashboards. Jenkins by default does not come with a built-in database. All the data gets stored in the filesystem. However, Jenkins has a REST API that allows us to read data as json. I searched around the Googlesphere to see if anyone had figured out how to capitalize on this REST API for performing analytics and found almost nothing directly related to this.

I was searching for open to public Jenkins instances to see how other people use Jenkins. I came across Elasticsearch's Jenkins site and I was pleasantly surprised. Elasticsearch is a json document storage application. There is a Jenkins plug-in that directly inputs data to Elasticsearch. 

I implemented this proof of concept to perform Jenkins analytics with Elasticsearch as the document store and Kibana as the analytics engine. I automated the installation part using Docker. There are better ways to automate this. The steps given below do not require any knowledge of Docker. 

Pre-requisites:

Vagrant
Oracle VirtualBox
Docker client installed on a running Ubuntu Vagrant box

Step 1: Create the docker containers

Execute the following docker commands to create the Docker containers needed for each of the apps.

#Create Elasticsearch container
docker run --name elasticsearch --hostname elasticsearch -p 9200:9200 -p 9300:9300 -d elasticsearch

#Create Jenkins container
docker run -p 8080:8080 -p 50000:50000 -v $HOME/jenkins_home:/var/jenkins_home --link elasticsearch:es --name jenkins -d jenkins

#Create Kibana container
docker run -e ELASTICSEARCH_URL=http://elasticsearch:9200 -p 5601:5601 --link elasticsearch:es --name kibana -d kibana


Step 2: Port forwarding from VirtualBox to host

Expose the following ports between the host computer and the virtual machine. Optionally, you may choose to add this to the Vagrantfile so that the port forwarding is automatically set whenever the Vagrantbox is started.

8080:8080

9200:9200

5601:5601


Step 3: Install the Jenkins Logstash plug-in

Click on Manage Jenkins and Manage Plug-ins



Click on 'Available' and Filter for 'Logstash'. Checkbox and click install without restart.

Step 4: Configure Jenkins to push data to Elasticsearch

Open http://localhost:8080/configure from a browser on the host computer. Look for the 'Logstash Plugin' section and fill out the following details.

Indexer type: ELASTICSEARCH
Host name: http://localhost
Port: 9200
Username: 
Password: 
Key: /logstash/type

Step 5: Create a Jenkins job and configure

Create a new jenkins job.


Check 'Send the console log to Logstash'.


Add a build step 'Execute shell'.


Step 6: Run Jenkins job

Run the job and see the console log.





Step 7: Verify data is getting to Elasticsearch

Open http://localhost:9200/logstash/_search?pretty in a browser and search for the jobname in the json.

Step 8: Get started with Kibana

I am still trying to figure out Kibana. Check out the visualize section and try to create a graph. I'll post that on another blogpost.

Open 'http://localhost:5601' in a browser and try it out.



Monday, November 23, 2015

Getting Started with @PuppetLabs - Part 4 - Supporting multiple operating systems

Most puppet modules can be designed to support multiple operating systems. Puppet has built in features to allow this through some simple syntax in the .pp file. It is a good practice to do that, whether you are publishing these modules on Puppet Forge or building highly customized modules for a specific purposes in a private enterprise.

1. Scalability - In case an organization chooses to switch operating systems or support other operating systems, your modules are already supporting them or they already have some operating systems and it is a matter of adding a few lines of code at the very top to add support for other operating systems.

2. Visibility and usability - The modules that support more than one operating system have higher usage counts on Puppet Forge because they support more than one type of need.

We will walk through a simple example to demonstrate multi-OS support. I checked in this code in GitHub at https://github.com/adityai/puppetApache.

Module: Install Apache web server and start it as a service.
OS Support: CentOs, Ubuntu and RHEL

Step 1: Folder structure
Create the following folder structure and blank files.

Folder: puppetApache
|-> Folder: apache -> Folder: manifests -> File: init.pp
|->  runner.pp

Step 2: Create runner.pp
We can use runner.pp to make it easy for us to apply the module from command line. Open the puppetApache/runner.pp file and enter the following line of code.

include apache

Step 3: Create the apache class
Open puppetApache/manifests/init.pp and enter the following code. I will explain each line of code in comments here.

#Create a class named apache
class apache {
#Case: when operating system is: case $::operatingsystem { #When operating system is ubuntu, set the package name and service name to apache2
"ubuntu": {
$packagename = "apache2"
$servicename = "apache2"
} #When operating system is centos or rhel, set the package name and service name to httpd
"centos", "rhel": {
$packagename = "httpd"
$sercicename = "httpd"
} #Default condition when the operating system is not supported, fail and stop.
default: {
fail("Ubsupported OS: ${::operatingsystem}")
}
}
#Ensure Apache is installed #Refer to puppet type reference https://docs.puppetlabs.com/references/latest/type.html#package
package { 'apache':
name => $packagename,
ensure => installed,
}
#Ensure Apache is running #Refer to puppet type reference https://docs.puppetlabs.com/references/latest/type.html#service
service { 'apache':
name => $servicename,
ensure => running, #require acts as a dependency. In this case, this service section requires the apache package section to be completed before this service section can run. #This ensures that dependencies are met before a particular section can execute.
require => Package['apache'],
}
}

Step #4: Run the module
Execute the following command from the puppetApache folder in the command line.

sudo puppet apply --modulepath . runner.pp

Step #5: Verify that apache is installed and running as a service
Execute the following command on the command line. You may have to run it with sudo depending on the user permissions.

For CentOs/RHEL:
service httpd status

For Ubuntu:
service apache2 status

Step #6: A little fun.
Now that we have confirmed that the apache web server is installed and running, let us stop the service and re-apply the puppet module.

Stop the Apache service
service apache2 stop #will stop the service on Ubuntu

Check the status of the service
service apache2 status #will show that the service is stopped

Apply the puppet module again.
sudo puppet apply --modulepath . runner.pp

Check the status of the service
service apache2 status #will show that the service is running

Here's what happened: When the puppet client applied the module:
1. It observed that apache is already installed (ensure => installed) and it does nothing with apache package.
2. It observed that the apache service is stopped (ensure => running) and it started the service.

Helpful Tip: If you really want your module to support multiple operating systems, I recommend setting up the case $::operatingsystem part as soon as you create the init.pp. As you add more sections, update the case $::operatingsystem part in parallel.

Have fun!



Saturday, November 14, 2015

Book review: #TheMartian by @AndyWeirAuthor



The story is gripping, exciting and everything about the atmosphere, trajectories, orbits and timing is scientifically accurate. The plot is about an astronaut who is stuck on the surface of Mars and how he survives and increases his chances for survival. The author also factored in the humanitarian aspect of how this kind of situation can unify people from all over our planet, their prayers and their willingness to help and sacrifice. I did not feel that the character, Mark Watney felt intense psychological stress, as one would expect, from being stranded on a deserted and inhospitable planet. There is some humor here and there that gave me a break from worrying for Mark Watney's situation. Some of them made me laugh really hard.

The story brought to life the fact that Mars is not a very hospitable place for us humans to be. I still wonder if it really makes sense to send humans to Mars. If we have to build a whole habitat for humans to live and survive the radiation, lack of atmosphere and lack of source of energy and food, why not just do it closer to home? How about our moon? I agree that we need to setup a backup plan for an accidental stray asteroid wiping us out, but why not start with the moon, perfect it and then head towards Mars and then may be later to Europa and Enceladus. I am talking about a 20 to 50 year timeframe for achieving something so grand and perfect. 

I really wish that Andy Weir made this book a lot less R rated. I was hoping to let my 8 year old wannabe Astrophysicist read this book, but I will have to wait a little longer before I let him read it because of occasional crude language. Andy Weir, hint ... hint .., can we edit the book and make a children friendly version? At least, get rid of the F and S words.

When I was done with this book, I remembered Carl Sagan's message from the original Cosmos documentary when he talked of the 'Pale blue dot'. I do not remember the exact words, but in essence it was about this - Earth is our only home, where everything that matters to everyone exists and we should cherish it, nurture it and protect it for ourselves and our future generations. 

I haven't seen the movie yet. I don't know if I can make time for it while it is in the theatre, but I promise to keep quiet and allow people who haven't read the book to enjoy the movie.


Thursday, November 12, 2015

@MongoDB Command line cheat sheet

Command line mongo

#Mongo Shell
mongo

Switch to a database

use test

List all databases

show dbs

List all collections

show collections
db.getCollectionNames()

Insert to the test database

db.restaurants.insert(
   {
      "address" : {
         "street" : "2 Avenue",
         "zipcode" : "10075",
         "building" : "1480",
         "coord" : [ -73.9557413, 40.7720266 ],
      },
      "borough" : "Manhattan",
      "cuisine" : "Italian",
      "grades" : [
         {
            "date" : ISODate("2014-10-01T00:00:00Z"),
            "grade" : "A",
            "score" : 11
         },
         {
            "date" : ISODate("2014-01-16T00:00:00Z"),
            "grade" : "B",
            "score" : 17
         }
      ],
      "name" : "Vella",
      "restaurant_id" : "41704620"
   }
)


Query for  all documents in a collection


db.restaurants.find()

 

Query with pretty

db.restaurants.find().pretty()

Query by a top level field

db.restaurants.find( { "borough": "Manhattan" } )

Query by a field in an embedded document

db.restaurants.find( { "address.zipcode": "10075" } )

Query by a field in an Array

db.restaurants.find( { "grades.grade": "B" } )

Query by operators

db.restaurants.find( { "grades.score": { $gt: 30 } } )
db.restaurants.find( { "grades.score": { $lt: 10 } } )

Query by logical conditions

Logical AND

db.restaurants.find( { "cuisine": "Italian", "address.zipcode": "10075" } )

Logical OR

db.restaurants.find(
   { $or: [ { "cuisine": "Italian" }, { "address.zipcode": "10075" } ] }
)

Sort query results

# 1 for ascending, -1 for descending
db.restaurants.find().sort( { "borough": 1, "address.zipcode": 1 } )

Update top level fields

db.restaurants.update(
    { "name" : "Vella" },
    {
      $set: { "cuisine": "American (New)" },
      $currentDate: { "lastModified": true }
    }
)

Update an embedded field

db.restaurants.update(
  { "restaurant_id" : "41704620" },
  { $set: { "address.street": "East 31st Street" } }
)

Update multiple documents

db.restaurants.update(
  { "address.zipcode": "10075", cuisine: "American (New)" },
  {
    $set: { cuisine: "Category To Be Determined" },
    $currentDate: { "lastModified": true }
  },
  { multi: true}
)

Replace a document

db.restaurants.update(
   { "restaurant_id" : "41704620" },
   {
     "name" : "Vella 2",
     "address" : {
              "coord" : [ -73.9557413, 40.7720266 ],
              "building" : "1480",
              "street" : "2 Avenue",
              "zipcode" : "10075"
     }
   }
)




Remove all documents that match a condition

db.restaurants.remove( { "name": "Vella 2" } )

Use the justOne option

#removes only one of the matching documents
db.restaurants.remove( { "borough": "Manhattan" }, { justOne: true } )

Remove all documents

db.restaurants.remove( { } )

Drop a collection

db.restaurants.drop()

Data Aggregation

Group documents by a field and calculate count

# Use the $group stage to group by a specified key. In the $group stage, specify the group by key in the _id field. $group accesses fields by the field path, which is the field name prefixed by a dollar sign $. The $group stage can use accumulators to perform calculations for each group. The following example groups the documents in the restaurants collection by the borough field and uses the $sum accumulator to count the documents for each group.
db.restaurants.aggregate(
   [
     { $group: { "_id": "$borough", "count": { $sum: 1 } } }
   ]
);

Filter and group documents

#Use the $match stage to filter documents. $match uses the MongoDB query syntax. The following pipeline uses $match to query the restaurants collection for documents with borough equal to "Manhattan" and cuisine equal to Italian. Then the $group stage groups the matching documents by the address.zipcode field and uses the $sum accumulator to calculate the count.
db.restaurants.aggregate(
   [
     { $match: { "borough": "Manhattan", "cuisine": "Italian" } },
     { $group: { "_id": "$address.zipcode" , "count": { $sum: 1 } } }
   ]);

Indexes

Create a single field index

db.restaurants.createIndex( { "cuisine": 1 } )

Create a compound index

# MongoDB supports compound indexes which are indexes on multiple fields. The order of the fields determine how the index stores its keys. For example, the following operation creates a compound index on the "cuisine" field and the "address.zipcode" field. The index orders its entries first by ascending "cuisine" values, and then, within each "cuisine", by descending "address.zipcode" values.
db.restaurants.createIndex( { "cuisine": 1, "address.zipcode": -1 } )

Unique constraint

#Just an example. Does not work with the restaurants collection.
db.members.createIndex( { "user_id": 1 }, { unique: true } )

FindOne

Find one document

#Find random document by calling findOne without any search parameters
db.people.findOne()

Find one specific document

#Find specific document by passing in one or more key values
db.people.findOne( {  “name” : “Smith” } )
db.people.findOne( { “name” : “Smith” , “age” : 30 } )

Find one document and display only the fields you want to see

#Find specific document and specify the fields you want to display. By default, _id is displayed.
db.people.findOne( { “name” : “Jones” } , { “name” :  true , “_id” : false } )

Query using $gt and $lt

Get all documents where a numeric value is greater than or less than a value

# $gt greater than
# $lt lesser than
# $gte greater than or equal to
# $lte lesser than or equal to

#Query by single condition
db.scores.find( { “score” : {  “$gt” : 95 } } )

#Query by condition and additional search values
db.scores.find( { “score” : {  “$gt” : 95 }  , “type” : “essay” } )

#Query by multiple conditions
db.scores.find( { “score” : {  “$gt” : 95 , “lte” : 98 } } )

Get all documents where a string is lexicographically greater than or less than another string

#Query for finding all documents where name is lexicographically greater than or equal to the letter D
db.people.find( { “name” : { $gt : “D” } } )

#Query for finding all documents where name is lexicographically greater than or equal to the letters Je
db.people.find( { “name” : { $gt : “Je” } } )

#Query for finding all documents where name is lexicographically greater than or equal to the letters Je and less than Jo
db.people.find( { "name" : { $gte : "Je" , "$lt" : "Jo" } } )

Exists

#Find all documents where the key profession exists
db.people.find( { “profession” : { $exists : true } } )

Type of a field

#Find all documents where name is a string value. $type of 2 is based on BSON specification.
db.people.find( { “name” : { $type : 2} }

Regular Expressions

#Query for string patterns
db.people.find( { "name" : { "$regex" : "A" } } )

#Query for a document where the string name ends in letter ‘y’
db.people.find( { "name" : { "$regex" : "y$" } } )

#Query for document where name begins with letter ‘A’
db.people.find( { "name" : { "$regex" : "^A" } } )

#Query for a document where name contains a letter ‘J’ and age exists
db.people.find( { "name" : { $regex : "J" } , "age" : { $exists : true } } );

$or

# Query to find all documents in people where name ends with an ‘e’ or age exists

db.people.find( { $or : [ { name : { $regex : "e$" } }, { age : { $exists : true } } ] } );

# Query to find all documents where score is greater than 90 or lesser than 50
db.scores.find( { $or : [ { score : { $gt : 90 } }, { score : { $lt : 50 } } ] } )

$and

#Find all people whose names lexicographically are greater than ‘A’ and the name also has a ‘n’ in it
db.people.find( { $and :[ { name : { $gt : "A" } }, { name : { $regex : "n" } } ] } )

#The same query can be performed without a $and. $and is not optimizable
db.people.find( { name : { $gt : "A", $regex : "n" } } )

Querying inside Arrays


> db.people.find()
{ "_id" : ObjectId("563ae19d4192c9f554e4293c"), "name" : "John" }
{ "_id" : ObjectId("563b8cabba646839a580a695"), "name" : "Jane", "age" : "30", "favorites" : [ "red", "green", "blue" ] }
{ "_id" : ObjectId("563b8cc2ba646839a580a696"), "name" : "Gerald", "age" : "32", "favorites" : [ "green", "blue" ] }

> db.people.find( { favorites : "red" } )
{ "_id" : ObjectId("563b8cabba646839a580a695"), "name" : "Jane", "age" : "30", "favorites" : [ "red", "green", "blue" ] }

> db.people.find( { favorites : "blue" } )
{ "_id" : ObjectId("563b8cabba646839a580a695"), "name" : "Jane", "age" : "30", "favorites" : [ "red", "green", "blue" ] }
{ "_id" : ObjectId("563b8cc2ba646839a580a696"), "name" : "Gerald", "age" : "32", "favorites" : [ "green", "blue" ] }

Using $all for searching in Array

# Matches any document that has all of the specified elements in the field irrespective of the order
> db.people.find( { favorites : { $all : ["blue", "green"] } } )
{ "_id" : ObjectId("563b8cabba646839a580a695"), "name" : "Jane", "age" : "30", "favorites" : [ "red", "green", "blue" ] }
{ "_id" : ObjectId("563b8cc2ba646839a580a696"), "name" : "Gerald", "age" : "32", "favorites" : [ "green", "blue" ] }

Querying for nth element in an Array



db.movieDetails.find({"countries.1": "Sweden"}).count()

Using $in for searching in Array

# Find all documents that have either of the values of name
> db.people.find( { name : { $in : [ "Andrew", "Andy" ] } } )
{ "_id" : ObjectId("563ae1894192c9f554e42939"), "name" : "Andrew" }
{ "_id" : ObjectId("563ae1944192c9f554e4293a"), "name" : "Andy" }

# Find all documents that have either red or blue as favorites
> db.people.find( { favorites : { $in : [ "red", "blue" ] } } )
{ "_id" : ObjectId("563b8cabba646839a580a695"), "name" : "Jane", "age" : "30", "favorites" : [ "red", "green", "blue" ] }
{ "_id" : ObjectId("563b8cc2ba646839a580a696"), "name" : "Gerald", "age" : "32", "favorites" : [ "green", "blue" ] }
{ "_id" : ObjectId("563b8dfdba646839a580a697"), "name" : "Geraldine", "age" : "32", "favorites" : "blue" }

Nested documents

Queries with dot notation

#Insert a nested document
> db.people.insert( { name : "Joe Biden", email : { work : "joe@whitehouse.gov", personal : "joe@joebiden.com" } } )
WriteResult({ "nInserted" : 1 })


#The query returns a result because the email document is a byte by byte match
> db.people.find( { email : { work : "joe@whitehouse.gov", personal : "joe@joebiden.com" } } )
{ "_id" : ObjectId("563b9070ba646839a580a698"), "name" : "Joe Biden", "email" : { "work" : "joe@whitehouse.gov", "personal" : "joe@joebiden.com" } }

#The query does not return anything because the order of elements in the email document is different and hence the byte by byte comparison does not find the document.
> db.people.find( { email : { personal : "joe@joebiden.com" }, work : "joe@whitehouse.gov" } )


#The query does not return anything because there is only one element in the search and the document actually has two elements.
> db.people.find( { email : { personal : "joe@joebiden.com" } } )

#The query returns a value because of the dot notation.
> db.people.find( { "email.personal" : "joe@joebiden.com" } )
{ "_id" : ObjectId("563b9070ba646839a580a698"), "name" : "Joe Biden", "email" : { "work" : "joe@whitehouse.gov", "personal" : "joe@joebiden.com" } }


#Quotations are necessary for dot notation to be syntactically correct.
> db.people.find( { email.personal : "joe@joebiden.com" } )
2015-11-05T17:25:17.789+0000 E QUERY    SyntaxError: Unexpected token .


Cursors

By default .find() method is actually creating a cursor object.
#Hold on to a cursor without printing out anything by using a null. This ensures that the cursor is held by cur and nothing gets printed.
> cur = db.people.find(); null;
null

hasNext()

> cur.hasNext()
True

next()

> cur.next()
{
        "_id" : ObjectId("563a40060e07998d8aaf2d28"),
        "name" : "Smith",
        "age" : 30,
        "profession" : "developer"
}
> cur.next()
{
        "_id" : ObjectId("563a40bf0e07998d8aaf2d29"),
        "name" : "Jones",
        "age" : 35
}

Iterate through the cursor

> while (cur.hasNext()) printjson(cur.next());
{ "_id" : ObjectId("563ae1894192c9f554e42939"), "name" : "Andrew" }
{ "_id" : ObjectId("563ae1944192c9f554e4293a"), "name" : "Andy" }
{ "_id" : ObjectId("563ae1984192c9f554e4293b"), "name" : "Jeff" }
{ "_id" : ObjectId("563ae19d4192c9f554e4293c"), "name" : "John" }
{
        "_id" : ObjectId("563b8cabba646839a580a695"),
        "name" : "Jane",
        "age" : "30",
        "favorites" : [
                "red",
                "green",
                "blue"
        ]
}
{
        "_id" : ObjectId("563b8cc2ba646839a580a696"),
        "name" : "Gerald",
        "age" : "32",
        "favorites" : [
                "green",
                "blue"
        ]
}
{
        "_id" : ObjectId("563b8dfdba646839a580a697"),
        "name" : "Geraldine",
        "age" : "32",
        "favorites" : "blue"
}
{
        "_id" : ObjectId("563b9070ba646839a580a698"),
        "name" : "Joe Biden",
        "email" : {
                "work" : "joe@whitehouse.gov",
                "personal" : "joe@joebiden.com"
        }
}

limit()

#Limits the number of documents in the cursor

> cur = db.people.find(); null;
null
> cur.limit(3)
{ "_id" : ObjectId("563a40060e07998d8aaf2d28"), "name" : "Smith", "age" : 30, "profession" : "developer" }
{ "_id" : ObjectId("563a40bf0e07998d8aaf2d29"), "name" : "Jones", "age" : 35 }
{ "_id" : ObjectId("563ae1894192c9f554e42939"), "name" : "Andrew" }





sort()

> cur = db.people.find(); null;
Null

#Sort the documents in the cursor in reverse order lexicographically by name
> cur.sort( { name : -1 } ); null;
Null

#Print the documents in the cursor iteratively
> while (cur.hasNext()) printjson(cur.next());
{
        "_id" : ObjectId("563a40060e07998d8aaf2d28"),
        "name" : "Smith",
        "age" : 30,
        "profession" : "developer"
}
{
        "_id" : ObjectId("563a40bf0e07998d8aaf2d29"),
        "name" : "Jones",
        "age" : 35
}
{ "_id" : ObjectId("563ae19d4192c9f554e4293c"), "name" : "John" }
{
        "_id" : ObjectId("563b9070ba646839a580a698"),
        "name" : "Joe Biden",
        "email" : {
                "work" : "joe@whitehouse.gov",
                "personal" : "joe@joebiden.com"
        }
}
{ "_id" : ObjectId("563ae1984192c9f554e4293b"), "name" : "Jeff" }
{
        "_id" : ObjectId("563b8cabba646839a580a695"),
        "name" : "Jane",
        "age" : "30",
        "favorites" : [
                "red",
                "green",
                "blue"
        ]
}
{
        "_id" : ObjectId("563b8dfdba646839a580a697"),
        "name" : "Geraldine",
        "age" : "32",
        "favorites" : "blue"
}
{
        "_id" : ObjectId("563b8cc2ba646839a580a696"),
        "name" : "Gerald",
        "age" : "32",
        "favorites" : [
                "green",
                "blue"
        ]
}
{ "_id" : ObjectId("563ae1944192c9f554e4293a"), "name" : "Andy" }
{ "_id" : ObjectId("563ae1894192c9f554e42939"), "name" : "Andrew" }

Chaining sort and limit

> cur = db.people.find(); null;
Null

#sort and limit chained
> cur.sort( { name: -1 } ).limit(2); null;
Null

#Print the contents of the cursor iteratively
> while (cur.hasNext()) printjson(cur.next());
{
        "_id" : ObjectId("563a40060e07998d8aaf2d28"),
        "name" : "Smith",
        "age" : 30,
        "profession" : "developer"
}
{
        "_id" : ObjectId("563a40bf0e07998d8aaf2d29"),
        "name" : "Jones",
        "age" : 35
}


Skip

 > cur = db.people.find(); null;
Null

#Skip chained to the sort and limit
> cur.sort( { name: -1 } ).limit(5).skip(2); null;
Null

#Print the cursor content iteratively
> while (cur.hasNext()) printjson(cur.next());
{ "_id" : ObjectId("563ae19d4192c9f554e4293c"), "name" : "John" }
{
        "_id" : ObjectId("563b9070ba646839a580a698"),
        "name" : "Joe Biden",
        "email" : {
                "work" : "joe@whitehouse.gov",
                "personal" : "joe@joebiden.com"
        }
}
{ "_id" : ObjectId("563ae1984192c9f554e4293b"), "name" : "Jeff" }
{
        "_id" : ObjectId("563b8cabba646839a580a695"),
        "name" : "Jane",
        "age" : "30",
        "favorites" : [
                "red",
                "green",
                "blue"
        ]
}
{
        "_id" : ObjectId("563b8dfdba646839a580a697"),
        "name" : "Geraldine",
        "age" : "32",
        "favorites" : "blue"
}

Counting results

Count command

> db.people.find( { name : {$gt : "A"} } )
{ "_id" : ObjectId("563a40060e07998d8aaf2d28"), "name" : "Smith", "age" : 30, "profession" : "developer" }
{ "_id" : ObjectId("563a40bf0e07998d8aaf2d29"), "name" : "Jones", "age" : 35 }
{ "_id" : ObjectId("563ae1894192c9f554e42939"), "name" : "Andrew" }
{ "_id" : ObjectId("563ae1944192c9f554e4293a"), "name" : "Andy" }
{ "_id" : ObjectId("563ae1984192c9f554e4293b"), "name" : "Jeff" }
{ "_id" : ObjectId("563ae19d4192c9f554e4293c"), "name" : "John" }
{ "_id" : ObjectId("563b8cabba646839a580a695"), "name" : "Jane", "age" : "30", "favorites" : [ "red", "green", "blue" ] }
{ "_id" : ObjectId("563b8cc2ba646839a580a696"), "name" : "Gerald", "age" : "32", "favorites" : [ "green", "blue" ] }
{ "_id" : ObjectId("563b8dfdba646839a580a697"), "name" : "Geraldine", "age" : "32", "favorites" : "blue" }
{ "_id" : ObjectId("563b9070ba646839a580a698"), "name" : "Joe Biden", "email" : { "work" : "joe@whitehouse.gov", "personal" : "joe@joebiden.com" } }

> db.people.count( { name : {$gt : "A"} } )
10

$set and $inc command

$set command

The command is used for setting specific values in a document using the update command. This command retains the data for all other fields in the document unlike the update command when used without the $set.

# Find a document where name is Jane
> db.people.find( { name : "Jane" } ).pretty()
{
        "_id" : ObjectId("563b8cabba646839a580a695"),
        "name" : "Jane",
        "age" : "30",
        "favorites" : [
                "red",
                "green",
                "blue"
        ]
}

# Set the age for Jane to 35
> db.people.update( { name : "Jane"}, { $set : { age : 35 }} )
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.people.find( { name : "Jane" } ).pretty()
{
        "_id" : ObjectId("563b8cabba646839a580a695"),
        "name" : "Jane",
        "age" : 35,
        "favorites" : [
                "red",
                "green",
                "blue"
        ]
}

$inc command

The command is used for incrementing a value in a document using the update command.

# Increment the age of Jane by one using $inc
> db.people.update( { name : "Jane"}, { $inc : { age : 1 }} )
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })

> db.people.find( { name : "Jane" } ).pretty()
{
        "_id" : ObjectId("563b8cabba646839a580a695"),
        "name" : "Jane",
        "age" : 36,
        "favorites" : [
                "red",
                "green",
                "blue"
        ]
}

$unset command

Removes the field and the value from a specified document that matches the search criteria.
> db.people.find( { name : "Jane" } ).pretty()
{
        "_id" : ObjectId("563b8cabba646839a580a695"),
        "name" : "Jane",
        "age" : 36,
        "favorites" : [
                "red",
                "green",
                "blue"
        ]
}

> db.people.update( { name : "Jane" }, { $unset : { favorites : 1 } } )
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })

> db.people.find( { name : "Jane" } ).pretty()
{
        "_id" : ObjectId("563b8cabba646839a580a695"),
        "name" : "Jane",
        "age" : 36
}


Array operations

> db.arrays.insert( { _id : 0 , a : [ 1, 2, 3, 4 ]} )
WriteResult({ "nInserted" : 1 })


> db.arrays.update( { _id : 0 }, { $set : { "a.2" : 5 } } )
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.arrays.find()
{ "_id" : 0, "a" : [ 1, 2, 5, 4 ] }


> db.arrays.update( { _id : 0 }, { $push : { a : 6 } } )
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.arrays.find()
{ "_id" : 0, "a" : [ 1, 2, 5, 4, 6 ] }


> db.arrays.update( { _id : 0 }, { $pop : { a : 1 } } )
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.arrays.update( { _id : 0 }, { $pop : { a : -1 } } )
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.arrays.find()
{ "_id" : 0, "a" : [ 2, 5, 4 ] }


> db.arrays.update( { _id : 0 }, { $pushAll : { a : [ 7, 8, 9, 10 ] } } )
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.arrays.find()
{ "_id" : 0, "a" : [ 2, 5, 4, 7, 8, 9, 10 ] }


> db.arrays.update( { _id : 0 }, { $pull : { a : 5 } } )
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.arrays.find()
{ "_id" : 0, "a" : [ 2, 4, 7, 8, 9, 10 ] }


> db.arrays.update( { _id : 0 }, { $pullAll : { a : [ 2, 7, 8 ] } } )
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.arrays.find()
{ "_id" : 0, "a" : [ 4, 9, 10 ] }


> db.arrays.update( { _id : 0 }, { $addToSet : { a : 5 } } )
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.arrays.update( { _id : 0 }, { $addToSet : { a : 5 } } )
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 0 })
> db.arrays.find()
{ "_id" : 0, "a" : [ 4, 9, 10, 5 ] }



Upsert command

Insert if the document searched for updating does not exist.
> db.people.find()
{ "_id" : ObjectId("563a40060e07998d8aaf2d28"), "name" : "Smith", "age" : 30, "profession" : "developer" }
{ "_id" : ObjectId("563a40bf0e07998d8aaf2d29"), "name" : "Jones", "age" : 35 }
{ "_id" : ObjectId("563ae1894192c9f554e42939"), "name" : "Andrew" }
{ "_id" : ObjectId("563ae1944192c9f554e4293a"), "name" : "Andy" }
{ "_id" : ObjectId("563ae1984192c9f554e4293b"), "name" : "Jeff" }
{ "_id" : ObjectId("563ae19d4192c9f554e4293c"), "name" : "John" }
{ "_id" : ObjectId("563b8cabba646839a580a695"), "name" : "Jane", "age" : 36 }
{ "_id" : ObjectId("563b8cc2ba646839a580a696"), "name" : "Gerald", "age" : "32", "favorites" : [ "green", "blue" ] }
{ "_id" : ObjectId("563b8dfdba646839a580a697"), "name" : "Geraldine", "age" : "32", "favorites" : "blue" }
{ "_id" : ObjectId("563b9070ba646839a580a698"), "name" : "Joe Biden", "email" : { "work" : "joe@whitehouse.gov", "personal" : "joe@joebiden.com" } }

#Update on a document that does not exist does nothing.
> db.people.update( { name : "Charlie" }, { $set : { age : 40 } } )
WriteResult({ "nMatched" : 0, "nUpserted" : 0, "nModified" : 0 })

#Update with a upsert on a document that does not exist, creates the document.
> db.people.update( { name : "Charlie" }, { $set : { age : 40 } }, { upsert : true }  )
WriteResult({
        "nMatched" : 0,
        "nUpserted" : 1,
        "nModified" : 0,
        "_id" : ObjectId("564420e692781bd83e60cd23")
})
> db.people.find( { name : "Charlie" })
{ "_id" : ObjectId("564420e692781bd83e60cd23"), "name" : "Charlie", "age" : 40 }

Multiupdate


#multi : true is required for updating all documents in a collection.
# {} in the search means ‘all documents in the collection’
> db.people.update( {}, { $set : { title : "Dr" } }, { multi : true } )
WriteResult({ "nMatched" : 11, "nUpserted" : 0, "nModified" : 11 })

> db.people.find()
{ "_id" : ObjectId("563a40060e07998d8aaf2d28"), "name" : "Smith", "age" : 30, "profession" : "developer", "title" : "Dr" }
{ "_id" : ObjectId("563a40bf0e07998d8aaf2d29"), "name" : "Jones", "age" : 35, "title" : "Dr" }
{ "_id" : ObjectId("563b8cabba646839a580a695"), "name" : "Jane", "age" : 36, "title" : "Dr" }
{ "_id" : ObjectId("563b8cc2ba646839a580a696"), "name" : "Gerald", "age" : "32", "favorites" : [ "green", "blue" ], "title" : "Dr" }
{ "_id" : ObjectId("563b8dfdba646839a580a697"), "name" : "Geraldine", "age" : "32", "favorites" : "blue", "title" : "Dr" }
{ "_id" : ObjectId("563b9070ba646839a580a698"), "name" : "Joe Biden", "email" : { "work" : "joe@whitehouse.gov", "personal" : "joe@joebiden.com" }, "title" : "Dr" }
{ "_id" : ObjectId("564420e692781bd83e60cd23"), "name" : "Charlie", "age" : 40, "title" : "Dr" }
{ "_id" : ObjectId("563ae1894192c9f554e42939"), "name" : "Andrew", "title" : "Dr" }
{ "_id" : ObjectId("563ae1944192c9f554e4293a"), "name" : "Andy", "title" : "Dr" }
{ "_id" : ObjectId("563ae1984192c9f554e4293b"), "name" : "Jeff", "title" : "Dr" }
{ "_id" : ObjectId("563ae19d4192c9f554e4293c"), "name" : "John", "title" : "Dr" }
> 


Remove


#Remove a specific document
db.people.remove( { name : “John” } )

#Remove documents one by one
db.people.remove( {} )

#Remove all documents in the collection
db.people.drop()


Distinct values

db.inventory.distinct( "dept" )