1. Network programming

Access to the network is as common nowadays, just like access to the local file system. Since Java 1.0 Java offers a network API for developing client-server applications. The Java library can establish encrypted connections and also brings support for the Hypertext Transfer Protocol (HTTP). The exercises in this chapter are about getting resources from a web server and developing a small client-server application with its protocol.

Prerequisites

  • know the URL class

  • know input/output streams

  • be able to read internet resources

  • be able to implement client and server with Socket and `ServerSocket

Data types used in this chapter:

1.1. URL and URLConnection

In Java, the obvious class URL represents a URL and URI a URI. An HTTP connection can be opened via URL, internally this is handled by the class URLConnection.

Both classes are not very convenient for modern HTTP calls; only in Java 11, a new package java.net.http has been added with HttpClient in the core. Java enterprise frameworks have more solutions, and there are many alternatives in the open-source universe as well:

  • Client API in Jakarta EE

  • WebClient in Spring Webflux

  • OkHttp

  • Apache HttpClient

  • Feign and Retrofit

1.1.1. Download remote images via URL ⭐

The URL class provides a method that returns an InputStream so that the bytes of the resource can be read.

Captain CiaoCiao likes to relax with pictures of well-shaped ships on shiphub.com. He would like to have some pictures on his storage device so that he has something to daydream about on long voyages.

Task:

  • For a given URL, write a program that downloads the resource and stores it on the local file system.

  • The file name should be based on the URL.

1.1.2. Read remote text file from URL ⭐

The Center for Systems Science and Engineering (CSSE) at Johns Hopkins University publishes a CSV file of covid disease data around the world every day at https://github.com/CSSEGISandData/COVID-19/tree/master/csse_covid_19_data/csse_covid_19_daily_reports. For March 1, 2021, the URL on the server is: https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_daily_reports/03-01-2021.csv.

The file starts with

FIPS,Admin2,Province_State,Country_Region,Last_Update,Lat,Long_,Confirmed,Deaths,Recovered,Active,Combined_Key,Incidence_Rate,Case-Fatality_Ratio
,,,Afghanistan,2020-10-22 04:24:27,33.93911,67.709953,40510,1501,33824,5185,Afghanistan,104.06300129769207,3.7052579609972844
,,,Albania,2020-10-22 04:24:27,41.1533,20.1683,17948,462,10341,7145,Albania,623.6708596844812,2.574102964118565
,,,Algeria,2020-10-22 04:24:27,28.0339,1.6596,55081,1880,38482,14719,Algeria,125.60932701190256,3.4131551714747372

Task:

  • Create a new class CoronaData with a new method String findByDateAndSearchTerm(LocalDate date, String search).

    • findByDateAndSearchTerm(…​) should build a URL object with the date. Note that the filename has the order month, day, year.

    • Open an input stream of the generated URL, read the stream line by line, and filter out all lines that do not contain the passed substring search.

    • The result is a string with all lines containing the search word, no need for a CSV parser.

Example:

  • The call findByDateAndSearchTerm( LocalDate.now().minusDays( 1 ), "Miguel" ) returns from the remote CSV document all Corona numbers from yesterday that contain "Miguel".

  • Sample return:

    8113,San Miguel,Colorado,US,2020-10-22 04:24:27,38.00450883,-108.4020725,100,0,0,100,"San Miguel, Colorado, US",1222.643354933366,0.0
    35047,San Miguel,New Mexico,US,2020-10-22 04:24:27,35.48014807,-104.8163562,151,0,0,151,"San Miguel, New Mexico, US",553.5799391428677,0.0

If the CORONA numbers go down, CSSE may stop publishing new documents. The old documents should stay.

1.2. HTTP client

Although URLConnection can be used to make an HTTP request, changing the HTTP method (GET, POST, PUT …​) and setting headers, this is not comfortable. Therefore, the HTTP client was added in Java 11. With the new API, HTTP resources can be obtained more elegantly over the network. Moreover, the HTTP client supports HTTP/1.1 and HTTP/2 as well as synchronous and asynchronous programming models. We will use this API to solve some exercises; those who cannot use Java 11 can find a similar library for Java 8 at https://github.com/AsyncHttpClient/async-http-client. In the next chapter on file formats, we will also return to the HTTP client in the exercises, since JSON or XML are often exchanged.

1.2.1. Top news from Hacker News ⭐⭐

Hacker News is a website with up-to-date discussions of technology trends. Articles can be accessed via a web service, and documentation can be found at https://github.com/HackerNews/API. Two endpoints are:

Task:

  • Create a new class HackerNews.

  • Implement a new method long[] hackerNewsTopStories() which will

  • Implement a new method String news(long id) that returns as a string the complete JSON document.

Example:

  • The class with the two methods can be used like this:

    System.out.println( Arrays.toString( hackerNewsTopStories() ) );
    String newsInJson = news( 24857356 );
    System.out.println( newsInJson );

1.3. Socket and ServerSocket

Operating systems provide sockets for TCP/UDP communication; Java can use them via the classes

  • java.net.Socket and java.net.ServerSocket for TCP and

  • java.net.DatagramSocket for UDP.

Objects of Socket and ServerSocket are created via the constructor or even better via the factories javax.net.SocketFactory and javax.net.ServerSocketFactory; for UDP there is no factory.

1.3.1. Implement a swear server and a client ⭐⭐

Bonny Brain is going to participate in the next swearing contest soon. She wants to prepare perfectly. A server should run an application that manages slurs, and clients can connect to this slur server and search for sentences.

Task:

  • Write a server and client.

  • Customize the server to accept multiple connections; use a thread pool.

  • The thread pool should use a maximum number of threads to prevent denial-of-service (DOS) attacks. If the maximum number of concurrent connections is exhausted, a client must wait until another connection becomes available.

Example:

  • After starting the server and client, an interaction may look like this:

    sir
    You, sir, are an oxygen thief!
    an
    You, sir, are an oxygen thief!
    Stop trying to be a smart ass, you're just an ass.

Solution SlangingMatchServer, link:solutions/src/main/java/com/tutego/exercise/net/SlangingMatchClient.java [solution SlangingMatchClient].

1.3.2. Implement a port scanner ⭐⭐

Bonny Brain installs the new Ay! OS, but important analysis tools are missing. It needs a tool that detects and reports the occupied TCP/UDP ports.

Task:

  • Write a program that tries to register a ServerSocket and DatagramSocket on all TCP/UDP ports from 0 to 49151; if it succeeds, the port is free. Otherwise it is busy.

  • Display the occupied ports on the console, and in addition, for the known ports, a description of the usual service that occupies that port.

Example:

  • The output might look like this:

    Protocol   Port       Service
    TCP         135       EPMAP
    UPD         137       NetBIOS Name Service
    UPD         138       NetBIOS Datagram Service
    TCP         139       NetBIOS Session Service
    TCP         445       Microsoft-DS Active Directory
    TCP         843       Adobe Flash
    UPD        1900       Simple Service Discovery Protocol (SSDP)
    UPD        3702       Web Services Dynamic Discovery
    TCP        5040
    UPD        5050
    UPD        5353       Multicast DNS
    UPD        5355       Link-Local Multicast Name Resolution (LLMNR)
    TCP        5939
    TCP        6463
    TCP        6942
    TCP       17500       Dropbox
    UPD       17500       Dropbox
    TCP       17600
    TCP       27017       MongoDB
    UPD       42420

A network interface connects computers via a computer network. In the following, we always assume a TCP/UDP interface. The network interface does not have to be physical, but can also be implemented in software. Like the loopback interface with the IP 127.0.0.1 (IPv4) or ::1 (IPv6). Typical network interfaces continue to exist for the LAN or WLAN. Operating system tools can display all network interfaces, such as ipconfig /all under Windows or ip a under Linux. In Java, the network interfaces can be retrieved via java.net.NetworkInterface. Each network interface has its IP address.

To register a server socket there are two possibilities: either the service only accepts requests from a special local InetAddress or the service accepts requests from all local addresses. So in principle, it is possible to bind the same port several times on one network card, because on one network card any number of network interfaces can be configured because are have distinguishable IP addresses.

The solution of the task can perform a simple test and register the socket on all network interfaces; if this fails, a service was already active on one of the network interfaces. This is enough for us as a criterion that on some network interface the port is busy.