[HER-1636] H3: Failed get of replay char sequence due the OutOfMemoryError: Map failed (port HER-1482)

(from nlevitt's H2 commit message:)
* InMemoryReplayCharSequence.java
    new, simple ReplayCharSequence that supports any encoding, keeping everything in memory
* GenericReplayCharSequence.java
    If the encoding supports random access, memory maps the backing file directly; otherwise decodes to UTF-16 and maps that. Supports the first Integer.MAX_VALUE bytes of the file. Maps up to 64M; moves the map around as necessary for larger files.
* Latin1ByteReplayCharSequence.java
    removed, functionality split between InMemoryReplayCharSequence and GenericReplayCharSequence
* ReplayCharSequenceTest.java
    xestHugeReplayCharSequence() - uncomment to test a huge replay char sequence
    testReplayCharSequenceByteToStringOverflow() - test both UTF-8 and windows-1252
* RecordingOutputStream.java
    use the new ReplayCharSequences
This commit is contained in:
gojomo committed 2009-07-07 21:53:57 +00:00
1 parent 1128ad638b
commit cb8a374e5d
6 files changed
+522 -665

No files matched your search

@@ -1,27 +1,22 @@
/* ReplayInputStream
/*
* This file is part of the Heritrix web crawler (crawler.archive.org).
*
* $Id$
* Licensed to the Internet Archive (IA) by one or more individual
* contributors.
*
* Created on Sep 24, 2003
* The IA licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* Copyright (C) 2003 Internet Archive.
* http://www.apache.org/licenses/LICENSE-2.0
*
* This file is part of the Heritrix web crawler (crawler.archive.org).
*
* Heritrix is free software; you can redistribute it and/or modify
* it under the terms of the GNU Lesser Public License as published by
* the Free Software Foundation; either version 2.1 of the License, or
* any later version.
*
* Heritrix is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser Public License for more details.
*
* You should have received a copy of the GNU Lesser Public License
* along with Heritrix; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.archive.io;
import java.io.File;
@@ -279,4 +274,9 @@ public class ReplayInputStream extends SeekInputStream
public long position() throws IOException {
return position;
}
// package private
byte[] getBuffer() {
return buffer;
}
}