Elasticsearch, Logstash, and Kibana, commonly referred to as ELK or the Elastic Stack, is a set of tools that can, well do a lot of things. It is most famous for its logging and analytics capabilities.
The sample code for this post can be found on GitHub.
The biggest bottleneck with I/O resources on the filesystem is the operating system, which controls access to the filesystem. Reading from, and writing to, the operating system, is much more expensive than storing data in memory, and that is the subject of this post: caching.
The sample code associated with this blog post can be found on GitHub.
The question “does compression actually help?” is admittedly pretty loaded. The real answer is sometimes, and it depends. I will not try to answer every use case, but I will provide a very specific example here that appears to provide a “probably not” answer (for this specific use case).
The sample code for this blog post can be found on GitHub.
A flavor I/O performance optimization that applies specifically to the filesystem is the decision on when to use Random Access instead of something like a BufferedInputStream. Random access allows for accessing a file in a similar way as a large array of bytes stored on the filesystem. From the oracle documentation on the RandomAccessFile class:
The sample code associated with this blog post can be found on GitHub.
Another potential source of I/O bottlenecks, across any medium, could be the process you choose to format the data in in the first place. For example, XML used to be a standard way to send information across the wire or store in a backend system, but the size overhead of XML as compared to JSON is about double (not to mention it’s somehow harder to read when formatted compared to JSON).
You can view the sample code associated with this blog post on GitHub.
While you can achieve massive improvements in I/O operations via buffering, another key part of tuning java code in general, which is applicable to I/O bound operations, is method call overhead. Methods that are unnecessarily called repeatedly can bog down operations.
You can view the sample code associated with this post on GitHub.
Now that we know how to benchmark using junit and jmh, let’s put it to the test and try to optimize some basic I/O operations. While filesystem tasks are much less common in the web-enabled world, understanding the basics can help us when we move on to streams across network connections.
You can view the sample code associated with this post on GitHub.
JMH is a lightweight code generator that can benchmark Java code. While many of the performance bottlenecks in today’s world are related to network calls and/or database queries, it’s still a good idea to understand the performance of our code at a lower level. In particular, by automating performance tests on our code, we can usually at least ensure that the performance was not accidentally made worse by some refactoring effort.
You can view the sample code associated with this post on Github
Using the static methods in the Files class, a member of the java.nio.file package, we can manipulate the file system reasonably easily.
The sample code for this repository can be found on Github.
System paths and file manipulation, usually within the java.nio package, in Java allow you to forgo some of the details related to streaming of files–which, while they offer low level details and optimization opportunities, typically take longer to develop and get right.
The sample code associated with this post can be found on Github.
A ZIP file format is a compressed, lossless archive of files of files and/or directories. While the content being compressed will matter as to the algorithm’s effectiveness, it is a common way to transfer files among peers, particularly on many windows machines.
