Interview question from EPI
https://github.com/epibook/epibook.github.io/blob/master/solutions/java/src/main/java/com/epi/Tail.java
public static String
tail(String fileName, int N) throws IOException {
RandomAccessFile filePtr = new RandomAccessFile(fileName, "r");
filePtr.seek(filePtr.length() - 1);
long fileSize = filePtr.length(), newLineCount = 0;
StringBuilder lastNLines = new StringBuilder();
// Reads file in reverse looking for '\n'.
for (long i = fileSize - 1; i != -1; i--) {
filePtr.seek(i);
int readByte = filePtr.readByte();
char c = (char) readByte;
if (c == '\n') {
++newLineCount;
if (newLineCount > N) {
break;
}
}
lastNLines.append(c);
}
close(filePtr);
lastNLines.reverse();
return lastNLines.toString();
}
http://stackoverflow.com/questions/557844/java-io-implementation-of-unix-linux-tail-f
Take a look at Apache Commons implementation of Tailer class. It does seem to handle log rotation as well.
If logrotation is done properly ('cp logfile oldfile; > logfile') then matt's solution should still work because the file reference isn't lost!
http://melandri.net/2009/05/29/java-tail/
http://stackoverflow.com/questions/20233950/tail-10-implementation-for-large-log-file-using-java
https://code.google.com/p/my-tail-implementation/source/browse/MyTail/src/tail/TailCommand.java?spec=svn4&r=4
Simple Tail for Java
https://github.com/dpillay/tail4j
https://github.com/maartenl/jtail/tree/master/src/com/tools/jtail
Apache commons-io Tailer
How to Implement tail -f in Java
http://fw-geekycoder.blogspot.com/2012/12/how-to-implement-tail-f-in-java.html
Java code to tail -N a text file
The technique is to simply map the file into memory using java.nio.channels.FileChannel.map in the Java NIO library and manipulate the file data using memory techniques.
For the "tail" function, we walk back the mapped bytes counting newlines. The MMapIterator class conveniently provides a way to iterate over lines once we find the starting line.
There is a point where care must be taken in the MMapIterator.next() implementation. That is making sure that bytes are converted to the appropriate string encoding. We use "UTF-8" but if you are dealing with a different encoding in the input file, this should be changed.
Java NIO
Java NIO allows reading huge files directly from the memory. i.e. The file need not be loaded into the JVM. All reads and writes on the byte buffer happen direclty on the file.
https://github.com/epibook/epibook.github.io/blob/master/solutions/java/src/main/java/com/epi/Tail.java
public static String
tail(String fileName, int N) throws IOException {
RandomAccessFile filePtr = new RandomAccessFile(fileName, "r");
filePtr.seek(filePtr.length() - 1);
long fileSize = filePtr.length(), newLineCount = 0;
StringBuilder lastNLines = new StringBuilder();
// Reads file in reverse looking for '\n'.
for (long i = fileSize - 1; i != -1; i--) {
filePtr.seek(i);
int readByte = filePtr.readByte();
char c = (char) readByte;
if (c == '\n') {
++newLineCount;
if (newLineCount > N) {
break;
}
}
lastNLines.append(c);
}
close(filePtr);
lastNLines.reverse();
return lastNLines.toString();
}
http://stackoverflow.com/questions/557844/java-io-implementation-of-unix-linux-tail-f
Take a look at Apache Commons implementation of Tailer class. It does seem to handle log rotation as well.
If logrotation is done properly ('cp logfile oldfile; > logfile') then matt's solution should still work because the file reference isn't lost!
http://melandri.net/2009/05/29/java-tail/
BufferedReader input = new BufferedReader(new FileReader(args[0]));
String currentLine = null;
while (true) {
if ((currentLine = input.readLine()) != null) {
System.out.println(currentLine);
continue;
}
try {
Thread.sleep(sleepTime);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
break;
}
}
input.close();
http://stackoverflow.com/questions/20233950/tail-10-implementation-for-large-log-file-using-java
https://code.google.com/p/my-tail-implementation/source/browse/MyTail/src/tail/TailCommand.java?spec=svn4&r=4
Simple Tail for Java
https://github.com/dpillay/tail4j
https://github.com/maartenl/jtail/tree/master/src/com/tools/jtail
Apache commons-io Tailer
How to Implement tail -f in Java
http://fw-geekycoder.blogspot.com/2012/12/how-to-implement-tail-f-in-java.html
public
static
void
open(File file) {
RandomAccessFile raf =
null
;
long
lastFilePointer =
0
;
raf =
new
RandomAccessFile(file,
"r"
);
while
(
true
) {
if
(raf.length() == lastFilePointer) {
// Don't forget to close the previous file handle
raf.close();
Thread.sleep(SLEEP_TIME);
// Wait till the file exists before opening it
while
(!file.exists()) {}
raf =
new
RandomAccessFile(file,
"r"
);
raf.seek(lastFilePointer);
}
else
{
byte
[] bytes =
new
byte
[
4096
];
int
bytesRead;
while
((bytesRead = raf.read(bytes,
0
, bytes.length)) != -
1
) {
System.out.print(
new
String(bytes,
0
, bytesRead));
}
lastFilePointer = raf.getFilePointer();
}
}
}
Java code to tail -N a text file
The technique is to simply map the file into memory using java.nio.channels.FileChannel.map in the Java NIO library and manipulate the file data using memory techniques.
For the "tail" function, we walk back the mapped bytes counting newlines. The MMapIterator class conveniently provides a way to iterate over lines once we find the starting line.
There is a point where care must be taken in the MMapIterator.next() implementation. That is making sure that bytes are converted to the appropriate string encoding. We use "UTF-8" but if you are dealing with a different encoding in the input file, this should be changed.
public class MMapFile { public class MMapIterator implements Iterator<String> { private int offset; public MMapIterator(int offset) { this.offset = offset; } public boolean hasNext() { return offset < cb.limit(); } public String next() { ByteArrayOutputStream sb = new ByteArrayOutputStream(); if (offset >= cb.limit()) throw new NoSuchElementException(); for (; offset < cb.limit(); offset++) { byte c = (cb.get(offset)); if (c == '\n') { offset++; break; } if (c != '\r') { sb.write(c); } } try { return sb.toString("UTF-8"); } catch (UnsupportedEncodingException e) {} return sb.toString(); } public void remove() { } } private ByteBuffer cb; long size; private long numLines = -1; public MMapFile(String file) throws FileNotFoundException, IOException { FileChannel fc = new FileInputStream(new File(file)).getChannel(); size = fc.size(); cb = fc.map(FileChannel.MapMode.READ_ONLY, 0, fc.size()); } public long getNumLines() { if (numLines != -1) return numLines; //cache number of lines long cnt = 0; for (int i=0; i <size; i++) { if (cb.get(i) == '\n') cnt++; } numLines = cnt; return cnt; } public Iterator<String> tail(long lines) { long cnt=0; long i=0; for (i=size-1; i>=0; i--) { if (cb.get((int)i) == '\n') { cnt++; if (cnt == lines+1) break; } } return new MMapIterator((int)i+1); } public Iterator<String> head() { return new MMapIterator(0); } static public void main(String[] args) { try { Iterator<String> it = new MMapFile("/test.txt").head(); while (it.hasNext()) { System.out.println(it.next()); } } catch (Exception e) { } System.out.println(); try { Iterator<String> it = new MMapFile("/test.txt").tail(2); while (it.hasNext()) { System.out.println(it.next()); } } catch (Exception e) { } System.out.println(); try { System.out.println("lines: "+new MMapFile("/test.txt").getNumLines()); } catch (Exception e) { } } }
Java NIO
Java NIO allows reading huge files directly from the memory. i.e. The file need not be loaded into the JVM. All reads and writes on the byte buffer happen direclty on the file.
MappedByteBuffer buffer = fileChannel.map(
FileChannel.MapMode.READ_ONLY,
0
, fileChannel.size());
// the buffer now reads the file as if it were loaded in memory. note
// that for smaller files it would be faster
// to just load the file in memory
// lets see if this buffer is loaded fully into memory
System.out.println(buffer.isLoaded());