dnsjava will create its global selector thread on first DNS lookup.
Since the first lookup occurs on a toe thread then this will cause the
selector thead to join the job's toe thread group. This is a problem
because when the job is terminated ToePool sends an interrupt to all
threads in the thread group which the dnsjava selector thread is
not expected and never clears. The interrupted state then causes
select() to return immediately instead of sleeping so it consumes
100% CPU.
So workaround this by doing a dummy lookup when FetchDNS is loaded,
this ensures the first lookup happens on the web server thread that
handles the job build action instead of a toe thread.
Fixes: 045b2516da ("Update to latest version of dnsjava")
Since the cache is global we only need to configure it once, not every
time we start a job. By moving it we keep the code that interfaces with
dnsjava centralized in FetchDNS.
Looks like dnsjava no longer uses a cleaner thread so remove the comment
referring to it.
The parsing throws StringIndexOutOfBoundsException when the version
does not contain '.' such as version "17". This check could never
actually trigger as the class cannot even be loaded on 1.7 or earlier,
so there's no reason to keep it.
Fixes#439
Colin reported this exception. I'm uncertain how this can occur though
as FetchHTTP should populate the response headers. Perhaps a different
Fetch module was used?
We hit a timeout during CI. The timeout is just a safety measure in case
the browser hangs so it doesn't hurt to have it higher. Using a higher
value will hopefully help if the system temporarily stalls for some
reason (garbage collection, IO issues, VM migration etc).
Each direct subclass of BaseResource defines an identical getEngine()
method so let's pull it up to BaseResource. We can also don't need the
type cast anymore as BaseResource.getApplication() does it for us.
Data URIs can be very large. ExtractorHTML mostly works with off-heap
CharSequences so by delaying the conversion of outlinks to strings
until after filtering out data URIs we can potentially avoid some
very large String allocations.
Alex Osborne