Task 2: Distributed Application and Dashboard The Distributed Application
Your application must be of your own creative design. (We will use software similarity detection software to identify those who do not.) It can be simple, but should fetch information from a 3rd party source and do something of at least marginal value. For example, we have assigned projects that generate hash values, implement clickers, or manage a blockchain. Your application should do something similarly simple but useful (but you should not reuse our ideas or the ideas of your peers!).
The following is a diagram of the components for your this part of your application:
Your web service should be deployed to the cloud and provide a simple RESTful API similar to those you have developed in prior projects. You do NOT have to implement all HTTP methods, only those that make sense for your application. Your web service must fetch information from some 3rd party API. In Project 1 you experimented with screen scraping, therefore that is not allowed in this project. Rather, you must find an API that provides data via XML or JSON. It is easy and can be fun to search for APIs; the GitHub Public APIs repository is a good place to start.
Use APIs that require authentication with caution. Many APIs will require you get a key (e.g. Flickr, which you used in the Android lab, required an API key). This is ok. But APIs that require authentication via OAuth or other schemes add a lot of work. Experiment ahead of time, but if you are brave, go ahead…
Be sure your API is from a reputable source. Your API still needs to be available when the TAs go to grade your project. Make sure you do not base your project on an API built by a 7th grade student…
Banned APIs: There are a number of APIs that have been used too often and are no longer interesting in this class. Therefore, you cannot use any of the following:
● Agify.io
● Alpha Vantage
● Eventful
● Flickr (for we have already done that)
● Google Maps (unless you also use a 2nd API to get info to put on the map)
● IsEvenAPI.xyz
● Merriam-Webster Dictionary
● NASA Astronomy Picture of the Day
● NYTimes APIs: specifically top stories, news wires, popular, and books.
● OpenMovieDatabase
● Pokemon API
● Spoonacular
● Any weather API
● Yahoo Finance API
Users will access your application via a native Android application. You do not need to have a browser-based interface for your application (only for the Dashboard). The Android application should communicate with your web service deployed to the cloud. Your web service is where the business logic for your application should be implemented (including fetching information from the 3rd party API).
In detail, your distributed application should satisfy the following requirements:
Distributed Application Requirements 1. Implement a native Android application
a. Has at least three different kinds of Views in your Layout (TextView, EditText, ImageView, or anything that extends android.view.View). In order to figure out if something is a View, find its API. If it extends android.view.View then it is a View. b. Requires input from the user
c. Makes an HTTP request (using an appropriate HTTP method) to your web service d. Receives and parses an XML or JSON formatted reply from your web service
e. Displays new information to the user
f. Is repeatable (I.e. the user can repeatedly reuse the application without restarting it.)
2. Implement a web service
a. Implement a simple (can be a single path) API.
b. Receives an HTTP request from the native Android application
c. Executes business logic appropriate to your application. This includes fetching XML or JSON information from some 3rd party API and processing the response.
● -10 if you use a banned API
● -10 if screen scrape instead of fetching XML or JSON via a published API
d. Replies to the Android application with an XML or JSON formatted response. The schema of the response can be of your own design.
● -5 if more information is returned to the Android app that is needed, forcing the mobile app to do more computing than is necessary. The web service should select and pass on only the information that the mobile app needs.
Use Servlets, not JAX-RS, for your web services. Students have had issues deploying web applications built with JAX-RS to Docker Containers and a solution has not yet been found.
Updated guidelines for deploying your web service to GitHub Codespaces will be provided before the Task 1 deadline.
3. Handle error conditions
Your application should test for and handle gracefully:
● Invalid mobile app input
● Invalid server-side input (regardless of mobile app input validation)
● Mobile app network failure, unable to reach server
● Third-party API unavailable
● Third-party API invalid data
Web Service Logging and Analysis Dashboard
Now enhance your web service to add logging, analysis, and reporting capabilities. In other words, create a web-based dashboard to your web service that will display information about how your service is being used. This will be web-page interface designed for laptop or desktop browser, not for mobile. In order to display logging and analytical data, you will have to first store it somewhere. For this task, you are required to store your data in a noSQL database, or more specifically a MongoDB, database hosted in the cloud.
The following is a diagram showing the dashboard components of your distributed application:
Logging data
Your web service should keep track (i.e. log) data regarding its use. You can decide what information would be useful to track for your web application, but you should track at least 6 pieces of information that would be useful for including in a dashboard for your application. It should include information about the request from the mobile phone, information about the request and reply to the 3rd party API, and information about the reply to the mobile phone. Information can include such parameters as what kind of model of phone has made the request, parameters included in the request specific to your application, timestamps for when requests are received, requests sent to the 3rd party API, and the data sent in the reply back to the phone. Be creative about what is useful for your application.
You should NOT log data from interaction with the operations dashboard, only from the mobile phone.
Storing logs
You should store your log data persistently so that it is available across restarts of the application. For this task you should use MongoDB to store your log data. MongoDB is a noSQL database that is easy to use. By incorporating it into your web service you will
gain experience using a noSQL database, and experience doing CRUD operations programmatically from a Java program to a database.
See the MongoDB section below for more information on MongoDB, how to set it up, and hints on how to connect to it.
The Dashboard
The purpose of logging data to the database is to be able to create an operations dashboard for your web service. This dashboard should be web page interface for use from a desktop or laptop browser (not a mobile device).
The dashboard should display two types of data:
1. Operationsanalytics–displayatleast3interestingoperationsanalyticsfrom your web service. You should choose analytics that are relevant to your specific web service. Examples for InterestingPicture might be top 10 picture search terms, average Flickr search latency, or the top 5 Android phone models making requests.
2. Logs–displaythedatalogsbeingstoredforeachmobilephoneuserinteraction with your web service. The display of each log entry can be simply formatted and should be easily readable. (Three points will be lost if they are displayed as JSON nor XML.)
You will likely find HTML tables useful for formatting tabular information on a web page. And there are plenty of examples of embedding data in tables with JSP on the web. No frameworks are necessary for this, just < 20 lines of JSP (i.e. mixed HTML and Java). You may use a client-side framework if you like (e.g. Twitter Bootstrap).
In detail, your dashboard should satisfy the additional requirements:
Web Service Logging and Analysis Dashboard Requirements
4. Log useful information
At least 6 pieces of information is logged for each request/reply with the mobile phone. It should include information about the request from the mobile phone, information about the request and reply to the 3rd party API, and information about the reply to the mobile phone. (You should NOT log data from interactions from the operations dashboard.)
5. Store the log information in a database
The web service can connect, store, and retrieve information from a MongoDB database in the cloud.
6. Display operations analytics and full logs on a web-based dashboard
a. A unique URL addresses a web interface dashboard for the web service. b. The dashboard displays at least 3 interesting operations analytics.
c. The dashboard displays formatted full logs.
7. Deploy the web service to GitHub Codespaces
In order to deploy your web application to the cloud using Github Cloudspaces, first make sure you have the basics working.
a. Accept the Github Classroom Assignment that you have been given the URL for. You will find a repository with:
● A .devcontainer.json and a Dockerfile which define how to create a Docker container, build a suitable software stack, and deploy the ROOT.war web application.
● A ROOT.war file which, like in Lab 3, contains a web application that will deployed in the container. This is a simple "Hello World!" application.
● An identical copy of this README.md
b. Click the green <> Code dropdown button, select the Codespaces tab, then click on
“Create codespace on master”.
c. Once the Codespace is running, the Terminal tab will show that Catalina (the Servlet container) is running. You should also see a “1” next to the Ports tab. Click on the Ports tab and you should see that port 8080 has been made available.
d. Mouse over the Local address item of the port 8080 line and you will find three icons. The leftmost is to copy the URL of your deployed application, the middle one (a globe) is to launch that URL in a browser. Clicking on the globe is a quick way to test your web service in a browswer. The copy is useful to use the URL in your Android App.
e. Click on the globe to confirm that the Hello World servlet is working.
f. By default, the URL in (d) requires you to be authenticated with Github. To test in a browser, that is fine, but when accessing your web service from your Android app, the
Programming Help, Add QQ: 749389476
Android app will not be authenticated. Therefore you must make the port visibility “Public”. To do this, right or control click on the word “Private” in the Visibility column, and change Port Visibility to “Public”. You will now be able to access the web service from your Android App or from an unauthenticated browser.
g. Copy the URL and paste into an Incognito Chrome window to confirm that the Hello World web app can be reached without authentication.
h. To deploy your own web service, create a ROOT.war like you did in Lab 3, upload or push the ROOT.war to your repository, and create a Codespace as has just been described.
The main MongoDB web site is https://www.mongodb.com. The site provides documentation, a downloadable version of the database manager application (mongod) that you can run on your laptop, and MongoDB drivers for many languages, including Java.
Mongod is the MongoDB database server. It listens by default on port 27017. Requests and responses with the database are made via a MongoDB protocol.
Mongo (without the DB) is a command line shell application for interacting with a MongoDB database. It is useful for doing simple operations on the database such as finding all the current contents or deleting them.
Because your web service will be running in the cloud, you can’t run your database on your laptop. Rather, you should use a MongoDB-as-a-Service to host your database in the cloud. Atlas (https://www.mongodb.com/atlas/database) is required because it has a free level of service that is adequate for your project.
Please read carefully… This project will challenge you to do a lot of research to understand enough MongoDB to create a simple database, add a collection, and insert, update, and find documents in that collection. This is very much like you will need to do regularly in industry. Code examples are provided on the MongoDB site, and elsewhere. As long as you include comments as to their source, you can use them in your code. If we search for a snippet of your code find it somewhere, and you have not attributed it to where you found it, that will be cheating and reason for receiving a failing grade in the course. Of course, the bulk of your code that is unique to your application should be your own and not copied from anywhere.
CS Help, Email: tutorcs@163.com
Setting up MongoDB Atlas
In this project, you are going to us nosql-database-as-a-service with MongoDB Atlas. Information about MongoDB can be found here: https://www.mongodb.com/what-is-mongodb
Getting started:
1. Createyouraccount.Gotohttps://www.mongodb.com/atlas/databaseandcreate your own free account.
2. Answerthe”Tellusafewthings…”questionshoweveryoulike,butincludeJava as the preferred language.
3. ChoosetocreateaFREEsharedcluster.
4. AcceptthedefaultsettingsandCreateCluster.
5. IntheSecurityQuickstart:
● How would you like to authenticate your connection?
Authenticate using Username and Password. Create a MongoDB user name and password (only use letters and numbers to save yourself some hassle for encoding it later) – don’t forget these. The cluster takes a few minutes to provision, so be patient.
● Where would you like to connect from?
Choose My Local Environment, add the IP address 0.0.0.0/0, and Add Entry. What this IP address means is that your DB will be open to the world, which is required for the grading purposes. (You can check this later on the Security tab, IP Whitelist. If it doesn’t have that IP address, click on Edit.)
● Then Finish and Close.
● If a popup window invites you to set up “Termination Protection”, just “Close”.
(You don’t need that for this project.)
6. Connecttothecluster.
a) Click on the Connect button.
b) Select the MongoDB Drivers option, then choose the Driver as Java, use version 4.3 or later.
c) Select Include full driver code example. Click Copy to copy that code stub. For now, save that code in a file; later, you’ll edit and paste into your application to connect to your MongoDB instance, but don’t forget to replace your with your database user‘s credentials (Note that when entering your password, any special characters are URL encoded; that’s why a simple password is better here).
d) You will access this database in two ways:
For Task 1: Create a simple Java application to demonstrate reading and writing to the database as described in Task 1 above.
For Task 2: In your Web Service Logging and Analysis Dashboard
The sample code in the Quick Start guide shows how to access the database. You can access this cloud-based MongoDB database from your laptop as well as from your web service in the cloud.
Info about the MongoDB Java driver and sample code can be found here:
https://docs.mongodb.com/drivers/java/sync/v4.3/quick-start/
You can easily add the MongoDB Java Drivers to a project with Maven:
Hints for connecting to MongoDB Atlas
Use a password that uses only letters and numbers so you don’t have to deal with encoding it.
The MongoDB connection string that Atlas provides is of the form:
tes=true&w=majority
But the +srv will not work with a number of DNS servers, and TLS and an authentication mechanism needs to be defined. Therefore:
1. OntheMongoDBAtlasdashboardwhereyoucreatedthedatabase,clickon Database and then on Cluster link (it might be Cluster0, Cluster1, etc.) Under the REGION label you will see three shard servers listed. Click on the name of each to display the full server URL, and copy the full URL. It will look something like cluster0-shard-00-02.cbkkm.mongodb.net:27017
2. FindtheURLsfortheothertwoshardserversandcopythemalso. 3. Createyourownconnectionstring:
s=true&tls=true&authMechanism=SCRAM-SHA-1
Be sure to substitute your own values for USER, PASSWD, and SERVER1-3
The resulting connection string should look similar to:
m.mongodb.net:27017,cluster0-shard-00-01.cbkkm.mongodb.net:
27017,cluster0-shard-00-00.cbkkm.mongodb.net:27017/myFirstD
atabase?w=majority&retryWrites=true&tls=true&authMechanism=
SCRAM-SHA-1
When running your application, you will see the following warning:
WARNING: SLF4J not found on the classpath. Logging is disabled for the
‘org.mongodb.driver’ component
If you would like to see the messages logged from the MongoDB driver, add the following dependencies to the pom.xml file: