1. Network programming
Access to the network is as common nowadays, just like access to the local file system. Since Java 1.0 Java offers a network API for developing client-server applications. The Java library can establish encrypted connections and also brings support for the Hypertext Transfer Protocol (HTTP). The exercises in this chapter are about getting resources from a web server and developing a small client-server application with its protocol.
Prerequisites
know the
URL
classknow input/output streams
be able to read internet resources
be able to implement client and server with
Socket
and `ServerSocket
Data types used in this chapter:
1.1. URL and URLConnection
In Java, the obvious class URL
represents a URL and URI
a URI. An HTTP connection can be opened via URL
, internally this is handled by the class URLConnection
.
Both classes are not very convenient for modern HTTP calls; only in Java 11, a new package java.net.http
has been added with HttpClient
in the core. Java enterprise frameworks have more solutions, and there are many alternatives in the open-source universe as well:
Client API in Jakarta EE
WebClient
in Spring WebfluxOkHttp
Apache HttpClient
Feign and Retrofit
1.1.1. Download remote images via URL ⭐
The URL
class provides a method that returns an InputStream
so that the bytes of the resource can be read.
Captain CiaoCiao likes to relax with pictures of well-shaped ships on shiphub.com. He would like to have some pictures on his storage device so that he has something to daydream about on long voyages.
Task:
For a given URL, write a program that downloads the resource and stores it on the local file system.
The file name should be based on the URL.
1.1.2. Read remote text file from URL ⭐
The Center for Systems Science and Engineering (CSSE) at Johns Hopkins University publishes a CSV file of covid disease data around the world every day at https://github.com/CSSEGISandData/COVID-19/tree/master/csse_covid_19_data/csse_covid_19_daily_reports. For March 1, 2021, the URL on the server is: https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_daily_reports/03-01-2021.csv.
The file starts with
FIPS,Admin2,Province_State,Country_Region,Last_Update,Lat,Long_,Confirmed,Deaths,Recovered,Active,Combined_Key,Incidence_Rate,Case-Fatality_Ratio ,,,Afghanistan,2020-10-22 04:24:27,33.93911,67.709953,40510,1501,33824,5185,Afghanistan,104.06300129769207,3.7052579609972844 ,,,Albania,2020-10-22 04:24:27,41.1533,20.1683,17948,462,10341,7145,Albania,623.6708596844812,2.574102964118565 ,,,Algeria,2020-10-22 04:24:27,28.0339,1.6596,55081,1880,38482,14719,Algeria,125.60932701190256,3.4131551714747372
Task:
Create a new class
CoronaData
with a new methodString findByDateAndSearchTerm(LocalDate date, String search)
.findByDateAndSearchTerm(…)
should build aURL
object with the date. Note that the filename has the order month, day, year.Open an input stream of the generated URL, read the stream line by line, and filter out all lines that do not contain the passed substring
search
.The result is a string with all lines containing the search word, no need for a CSV parser.
Example:
The call
findByDateAndSearchTerm( LocalDate.now().minusDays( 1 ), "Miguel" )
returns from the remote CSV document all Corona numbers from yesterday that contain"Miguel"
.Sample return:
8113,San Miguel,Colorado,US,2020-10-22 04:24:27,38.00450883,-108.4020725,100,0,0,100,"San Miguel, Colorado, US",1222.643354933366,0.0 35047,San Miguel,New Mexico,US,2020-10-22 04:24:27,35.48014807,-104.8163562,151,0,0,151,"San Miguel, New Mexico, US",553.5799391428677,0.0
If the CORONA numbers go down, CSSE may stop publishing new documents. The old documents should stay. |
1.2. HTTP client
Although URLConnection
can be used to make an HTTP request, changing the HTTP method (GET, POST, PUT …) and setting headers, this is not comfortable. Therefore, the HTTP client was added in Java 11. With the new API, HTTP resources can be obtained more elegantly over the network. Moreover, the HTTP client supports HTTP/1.1 and HTTP/2 as well as synchronous and asynchronous programming models. We will use this API to solve some exercises; those who cannot use Java 11 can find a similar library for Java 8 at https://github.com/AsyncHttpClient/async-http-client. In the next chapter on file formats, we will also return to the HTTP client in the exercises, since JSON or XML are often exchanged.
1.2.1. Top news from Hacker News ⭐⭐
Hacker News is a website with up-to-date discussions of technology trends. Articles can be accessed via a web service, and documentation can be found at https://github.com/HackerNews/API. Two endpoints are:
https://hacker-news.firebaseio.com/v0/topstories.json: returns a JSON array with IDs of the most discussed articles.
https://hacker-news.firebaseio.com/v0/item/24857356.json: returns a JSON object containing the message with ID
24857356
.
Task:
Create a new class
HackerNews
.Implement a new method
long[] hackerNewsTopStories()
which willconnects with the HTTP client to the endpoint https://hacker-news.firebaseio.com/v0/topstories.json,
splits the return from the JSON into long,
returns all IDs as
long[]
.In case of an IO error, an empty array should be returned.
Implement a new method
String news(long id)
that returns as a string the complete JSON document.
Example:
The class with the two methods can be used like this:
System.out.println( Arrays.toString( hackerNewsTopStories() ) ); String newsInJson = news( 24857356 ); System.out.println( newsInJson );
1.3. Socket and ServerSocket
Operating systems provide sockets for TCP/UDP communication; Java can use them via the classes
java.net.Socket
andjava.net.ServerSocket
for TCP andjava.net.DatagramSocket
for UDP.
Objects of Socket
and ServerSocket
are created via the constructor or even better via the factories javax.net.SocketFactory
and javax.net.ServerSocketFactory
; for UDP there is no factory.
1.3.1. Implement a swear server and a client ⭐⭐
Bonny Brain is going to participate in the next swearing contest soon. She wants to prepare perfectly. A server should run an application that manages slurs, and clients can connect to this slur server and search for sentences.
Task:
Write a server and client.
Customize the server to accept multiple connections; use a thread pool.
The thread pool should use a maximum number of threads to prevent denial-of-service (DOS) attacks. If the maximum number of concurrent connections is exhausted, a client must wait until another connection becomes available.
Example:
After starting the server and client, an interaction may look like this:
sir You, sir, are an oxygen thief! an You, sir, are an oxygen thief! Stop trying to be a smart ass, you're just an ass.
Solution SlangingMatchServer, link:solutions/src/main/java/com/tutego/exercise/net/SlangingMatchClient.java [solution SlangingMatchClient].
1.3.2. Implement a port scanner ⭐⭐
Bonny Brain installs the new Ay! OS, but important analysis tools are missing. It needs a tool that detects and reports the occupied TCP/UDP ports.
Task:
Write a program that tries to register a
ServerSocket
andDatagramSocket
on all TCP/UDP ports from 0 to 49151; if it succeeds, the port is free. Otherwise it is busy.Display the occupied ports on the console, and in addition, for the known ports, a description of the usual service that occupies that port.
Example:
The output might look like this:
Protocol Port Service TCP 135 EPMAP UPD 137 NetBIOS Name Service UPD 138 NetBIOS Datagram Service TCP 139 NetBIOS Session Service TCP 445 Microsoft-DS Active Directory TCP 843 Adobe Flash UPD 1900 Simple Service Discovery Protocol (SSDP) UPD 3702 Web Services Dynamic Discovery TCP 5040 UPD 5050 UPD 5353 Multicast DNS UPD 5355 Link-Local Multicast Name Resolution (LLMNR) TCP 5939 TCP 6463 TCP 6942 TCP 17500 Dropbox UPD 17500 Dropbox TCP 17600 TCP 27017 MongoDB UPD 42420
A network interface connects computers via a computer network. In the following, we always assume a TCP/UDP interface. The network interface does not have to be physical, but can also be implemented in software. Like the loopback interface with the IP To register a server socket there are two possibilities: either the service only accepts requests from a special local The solution of the task can perform a simple test and register the socket on all network interfaces; if this fails, a service was already active on one of the network interfaces. This is enough for us as a criterion that on some network interface the port is busy. |