Package org.apache.hadoop.util

Class WeakReferenceMap<K,V>

java.lang.Object

org.apache.hadoop.util.WeakReferenceMap<K,V>

Direct Known Subclasses:

WeakReferenceThreadMap

@Private
public class WeakReferenceMap<K,V>
extends java.lang.Object

A map of keys type K to objects of type V which uses weak references, so does lot leak memory through long-lived references at the expense of losing references when GC takes place.. This class is intended be used instead of ThreadLocal storage when references are to be cleaned up when the instance holding. In this use case, the key is the Long key. Concurrency. The class assumes that map entries are rarely contended for when writing, and that not blocking other threads is more important than atomicity. - a ConcurrentHashMap is used to map keys to weak references, with all its guarantees. - there is no automatic pruning. - see create(Object) for the concurrency semantics on entry creation.

Constructor Summary

Constructors

Constructor	Description
WeakReferenceMap(java.util.function.Function<? super K,? extends V> factory, java.util.function.Consumer<? super K> referenceLost)	instantiate.

Method Summary

Modifier and Type	Method	Description
void	clear()	Clear all entries.
boolean	containsKey(K key)	Does the map have a valid reference for this object? no-side effects: there's no attempt to notify or cleanup if the reference is null.
V	create(K key)	Create a new instance under a key.
V	get(K key)	Get the value, creating if needed.
long	getEntriesCreatedCount()	Get count of entries created on demand.
long	getReferenceLostCount()	Get count of references lost as detected during prune() or get() calls.
java.lang.ref.WeakReference<V>	lookup(K key)	look up the value, returning the possibly empty weak reference to a value, or null if no value was found.
int	prune()	Prune all null weak references, calling the referenceLost callback for each one.
V	put(K key, V value)	Put a value under the key.
V	remove(K key)	Remove any value under the key.
protected V	resolve(java.lang.ref.WeakReference<V> r)	Given a possibly null weak reference, resolve its value.
int	size()	Map size.
java.lang.String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

WeakReferenceMap

public WeakReferenceMap(java.util.function.Function<? super K,? extends V> factory,
                        @Nullable
                        java.util.function.Consumer<? super K> referenceLost)

instantiate.

Parameters:

factory - supplier of new instances

referenceLost - optional callback on lost references.

Method Detail

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

size

public int size()

Map size.

Returns:

the current map size.

clear

public void clear()

Clear all entries.

lookup

public java.lang.ref.WeakReference<V> lookup(K key)

look up the value, returning the possibly empty weak reference to a value, or null if no value was found.

Parameters:

key - key to look up

Returns:

null if there is no entry, a weak reference if found

get

public V get(K key)

Get the value, creating if needed.

Parameters:

key - key.

Returns:

an instance.

create

public V create(K key)

Create a new instance under a key.

The instance is created, added to the map and then the map value retrieved. This ensures that the reference returned is that in the map, even if there is more than one entry being created at the same time. If that race does occur, it will be logged the first time it happens for this specific map instance.

HADOOP-18456 highlighted the risk of a concurrent GC resulting a null value being retrieved and so returned. To prevent this:

1. A strong reference is retained to the newly created instance in a local variable.
2. That variable is used after the resolution process, to ensure the JVM doesn't consider it "unreachable" and so eligible for GC.
3. A check is made for the resolved reference being null, and if so, the put() is repeated

Parameters:

key - key

Returns:

the created value

put

public V put(K key,
             V value)

Put a value under the key. A null value can be put, though on a get() call a new entry is generated

Parameters:

key - key

value - value

Returns:

any old non-null reference.

remove

public V remove(K key)

Remove any value under the key.

Parameters:

key - key

Returns:

any old non-null reference.

containsKey

public boolean containsKey(K key)

Does the map have a valid reference for this object? no-side effects: there's no attempt to notify or cleanup if the reference is null.

Parameters:

key - key to look up

Returns:

true if there is a valid reference.

resolve

protected V resolve(java.lang.ref.WeakReference<V> r)

Given a possibly null weak reference, resolve its value.

Parameters:

r - reference to resolve

Returns:

the value or null

prune

public int prune()

Prune all null weak references, calling the referenceLost callback for each one. non-atomic and non-blocking.

Returns:

the number of entries pruned.

getReferenceLostCount

public final long getReferenceLostCount()

Get count of references lost as detected during prune() or get() calls.

Returns:

count of references lost

getEntriesCreatedCount

public final long getEntriesCreatedCount()

Get count of entries created on demand.

Returns:

count of entries created