par::mem::chunk< T > Class Template Reference

A par::mem::chunk is a simple handle to a part of data defined by an offset and a length of what part should be mapped. More...

#include <par_mem_chunk.h>

Inheritance diagram for par::mem::chunk< T >:

Inheritance graph
[legend]
Collaboration diagram for par::mem::chunk< T >:

Collaboration graph
[legend]

List of all members.

Public Member Functions

 chunk (FILE *of, off_t offs, off_t len) throw (sys::error)
 Link into an object given by a FILE buffer.
 chunk (int fd, off_t offs, off_t len) throw (sys::error)
 Link into an object given by a file descriptor.
 chunk (char const *URL) throw (sys::error)
 Create a chunk that has a permanent location in URL.
 chunk (char const *URL, off_t offs) throw (sys::error)
 Create a chunk that has a permanent location in URL.
 chunk (char const *URL, off_t offs, off_t len) throw (sys::error)
 Create a chunk that has a permanent location in URL.
 chunk (void) throw (sys::error)
 Create a chunk of length 0.
 chunk (off_t size) throw (sys::error)
 Contruction of a chunk that will correspond to a newly created memory segment. Your only way to acquire large parts of memory.
 chunk (chunk const &others, off_t off, off_t len) throw (sys::error)
 Construct a chunk that gets a window on the memory that other points to. See the general discussion of chunk for more explanation.
 chunk (chunk const &other) throw (sys::error)
chunk< T > & deepcopy (T *there, off_t amount)
chunk< T > & deepcopy (par::mem::chunk< T const > &there)
chunk duplicate (chunk< T const > const &other)
 Let this par::mem::chunk point to exactly the same window of other.
bool extend (off_t nsize) throw (sys::error)
T * get (void) const
T volatile * get_volatile (void) const
T const * getro (void) const
T volatile const * getro_volatile (void) const
off_t Length () const
void link (char const *URL)
 Let this par::mem::chunk point to exactly the same whole object refered to by other.
T * map (void)
 Map the chunk writable (if not yet done) and return a pointer to the memory.
T volatile * map_volatile (void)
T const * mapro (void)
 Map the chunk readonly (if not yet done) and return a pointer to the memory.
T volatile const * mapro_volatile (void)
off_t Offset () const
 operator chunk< T const > & (void)
 Const casting a chunk.
 operator chunk< T const >const & (void) const
template<class oT >
bool operator!= (chunk< oT > const &other) const
 The negation of both types of operator==.
chunkoperator<< (par::mem::chunk< T const > &there)
 An alias for par::mem::chunk<T>::deepcopy().
chunkoperator= (chunk const &other)
template<class oT >
bool operator== (chunk< oT > const &other) const
 Two chunks of different type will never be considered the same.
bool operator== (chunk< T > const &other) const
 Two chunks over the same T type are the same if they point to the same heap and have same offset and length. No dynamic type checking is necessary.
void prefetch (void)
par::mem::chunk< T > * split (size_t s, off_t const *const cnts, par::mem::chunk< T > *ret) const
 Split chunk into chunks of given sizes.
par::mem::chunk< T > * split (size_t s, off_t const *const cnts) const
 Split chunk into chunks of given sizes.
apointer< T > temp (off_t off) const
apointer< T > temp (off_t off, off_t len) const
 Create a temporary to be used in a expression that expects a T* or similar.
apointer< T > temp (void) const
bool truncate (off_t nsize) throw (sys::error)
void unmap (void)
 Unmap a chunk previously mapped with map(). Produces an error if memory is not mapped.
void unmapro (void)
 Unmap a chunk previously mapped with mapro(). Produces an error if memory is not mapped.
char const * url (void)
chunk window (char const *URL, off_t off, off_t len) throw (sys::error)
 Let this par::mem::chunk point to a window inside the object refered by URL.
chunkwindow (chunk< T const > const &other, off_t off)
chunkwindow (chunk< T const > const &other, off_t off, off_t len)
 Let this par::mem::chunk point to a window inside the window of other.
 ~chunk (void)

Static Public Member Functions

static void static_init (void)

Friends

req::contextreq::context::generate (size_t sessionId, size_t sessionSize, size_t mynum)


Detailed Description

template<class T>
class par::mem::chunk< T >

A par::mem::chunk is a simple handle to a part of data defined by an offset and a length of what part should be mapped.

The data can live in a memory segment (acquired by malloc or shm_open), in a file or whatever the future may bring.

Chunks are typed by the template parameter T. E.g

     par::mem::chunk< int > A(100);
     par::mem::chunk< double > B(100);
     par::mem::chunk< MyFavoriteType > C(100);
define three chunks of one hundred elements each, but of different T types. So the total memory that is reserved for them will be different (and platform dependent).

A par::mem::chunk is not mapped on construction, you cannot use the memory that it is pointing to directly. Usually you will access the data by means of the class par::mem::apointer or via the macro PAR_MEM_CHUNK_POINTER. This will ensure that the data is mapped into adress space and can then be used almost like a normal C pointer. Afterwards, on destruction, it also does the unmapping of the memory for you.

Think of a par::mem::chunk more as a being kind of handle of data than of the underlying memory itself. The memory models may vary at lot between different runs of the same program. The particular memory for a variable may even not always be mapped into the adress space of some parXXL process. E.g temporarily the data might be completely written to disk or other permanent storage.

For the memory model it is also very important that you distinguish readonly and write access to data. Beware,

 par::mem::chunk< ulong const > a(100);
 par::mem::chunk< ulong > c(100);
are not the same. The first gives you readonly access, the second read-write. Use the const version whenever you can:

Chunks can be constructed from scratch ( par::mem::chunk::chunk(off_t) ), from other objects (files (par::mem::file) or memory segments (par::mem::segment)), or just from another par::mem::chunk.

Files or memory segments can be linked by a simple call to the constructor par::mem::chunk::chunk(char const*). This lets you handle files in a very convenient way:

 par::mem::chunk< ulong > c;
 {
   par::mem::chunk< ulong const > myInFile("file::///tmp/bla.in");
   c.truncate(myInFile.Length());
   myInFile >> c;
 }

 do something with c

 {
  par::mem::chunk< ulong > myOutFile("file::///tmp/bla.out", 0, c.Length());
   myOutFile << c;
 }
This reads data from a file (possibly with readonly access) and writes to another file at the end.

If you create from such a name of an object but which object does not exist, the behavior is different according to whether or not your base type T has the const attribute. If it has it and the given name is a valid name that is accessible to the program (e.g file access rights in the corresponding directory are correct), the object is not created and an empty par::mem::chunk is returned. In particular, this is not considered an error and no exception is thrown. Test for such a situation with Length(). You may then yourself throw an exception, if your program can't cope with that situation.

If T does not have the const attribute the object is created if possible. If not possible, an exception of type par::sys::error is thrown, which you might catch to implement a fall back strategie, such as asking the user for a different name.

Chunks can be constructed from another by also specifying an other interval on the same memory segment. But, a given par::mem::chunk can only propagate the interval to which it corresponds (or part of it). As an example consider the following snipset:

     par::mem::chunk< int > A(100);
     par::mem::chunk< int > B(A, 10, 23);
     par::mem::chunk< int > C(B, 9, 14);
This defines A having 100 int's. B has a length of 23 corrsponding to A[10] ... A[32] and C has 14 int's corresponding to B[9] ... B[22] which in turn are A[19] ... A[32]. Trying to construct a par::mem::chunk from some other such that the range of the new one would go outside the range of the old one is an error.

See also:
par::step::array for a class that lets you use chunks in a consistent way amoung the different parXXL processes.

par::mem::req for the underlying mechanism that implements access to different types of Handling memory..

Definition at line 155 of file par_mem_chunk.h.


Constructor & Destructor Documentation

template<class T>
par::mem::chunk< T >::chunk ( chunk< T > const &  other  )  throw (sys::error) [inline]

Definition at line 186 of file par_mem_chunk.h.

template<class T>
par::mem::chunk< T >::chunk ( chunk< T > const &  others,
off_t  off,
off_t  len 
) throw (sys::error) [inline]

Construct a chunk that gets a window on the memory that other points to. See the general discussion of chunk for more explanation.

Definition at line 267 of file par_mem_chunk.h.

template<class T>
par::mem::chunk< T >::chunk ( off_t  size  )  throw (sys::error) [inline]

Contruction of a chunk that will correspond to a newly created memory segment. Your only way to acquire large parts of memory.

Definition at line 276 of file par_mem_chunk.h.

template<class T>
par::mem::chunk< T >::chunk ( void   )  throw (sys::error) [inline]

Create a chunk of length 0.

  chunk< double > principal(100);
  chunk< double > parts[2];
  part[0].window(principal, 0, 50);
  part[1].window(principal, 50, 50);
In this example the two chunks of the array parts, parts[0] and parts[1], are first constructed with this constructor without arguments. Then the invocations of window() link both as different sub-windows of principal.

Definition at line 296 of file par_mem_chunk.h.

template<class T>
par::mem::chunk< T >::chunk ( char const *  URL,
off_t  offs,
off_t  len 
) throw (sys::error) [inline]

Create a chunk that has a permanent location in URL.

If T has the const attribute, the object is accessed read-only. If in that case URL does not exist, it is not created but an empty par::mem::chunk is returned. Test for such a situation with Length().

Definition at line 310 of file par_mem_chunk.h.

template<class T>
par::mem::chunk< T >::chunk ( char const *  URL,
off_t  offs 
) throw (sys::error) [inline]

Create a chunk that has a permanent location in URL.

This is equivalent to par::mem::chunk::chunk(char const*, off_t, off_t) with a window starting at position offs and ending at the current end of the file.

If T has the const attribute, the object is accessed read-only. If in that case URL does not exist, it is not created but an empty par::mem::chunk is returned. Test for such a situation with Length().

Definition at line 327 of file par_mem_chunk.h.

template<class T>
par::mem::chunk< T >::chunk ( char const *  URL  )  throw (sys::error) [inline]

Create a chunk that has a permanent location in URL.

This is equivalent to par::mem::chunk::chunk(char const*, off_t, off_t) with a window corresponding to the whole file.

If T has the const attribute, the object is accessed read-only. If in that case URL does not exist, it is not created but an empty par::mem::chunk is returned. Test for such a situation with Length().

Definition at line 342 of file par_mem_chunk.h.

template<class T>
par::mem::chunk< T >::chunk ( int  fd,
off_t  offs,
off_t  len 
) throw (sys::error) [inline]

Link into an object given by a file descriptor.

Definition at line 350 of file par_mem_chunk.h.

template<class T>
par::mem::chunk< T >::chunk ( FILE *  of,
off_t  offs,
off_t  len 
) throw (sys::error) [inline]

Link into an object given by a FILE buffer.

Definition at line 359 of file par_mem_chunk.h.

template<class T>
par::mem::chunk< T >::~chunk ( void   )  [inline]

Definition at line 367 of file par_mem_chunk.h.


Member Function Documentation

template<class T>
chunk<T>& par::mem::chunk< T >::deepcopy ( T *  there,
off_t  amount 
) [inline]

Definition at line 535 of file par_mem_chunk.h.

template<class T>
chunk<T>& par::mem::chunk< T >::deepcopy ( par::mem::chunk< T const > &  there  )  [inline]

Definition at line 528 of file par_mem_chunk.h.

template<class T>
chunk par::mem::chunk< T >::duplicate ( chunk< T const > const &  other  )  [inline]

Let this par::mem::chunk point to exactly the same window of other.

Returns a reference to the calling object, to be compatible with par::mem::duplicate.

Definition at line 241 of file par_mem_chunk.h.

template<class T>
bool par::mem::chunk< T >::extend ( off_t  nsize  )  throw (sys::error) [inline]

Extend the size of the underlying object if possible.

Use with care and always check the return value.

Precondition:
  • This chunk must have write access to the object.
  • The object must not be mapped by this chunk.
  • The chunk must link to the entire object, i.e Offset() must be zero and Length() must be the length of the object.
Warning:
Any lock that might be held by somebody external to parXXL (eg. if the object is a file) may block this parXXL process.
Returns:
value of true means the call is on error.
Any other chunk that was previously refering to the entire object, will not do so after a successful call to extend. So such a chunk by consequence will loose the capacity to call truncate or extend successfully. On the other hand mappings that such a chunk had previously should be unaffected and continue to be valid.

This call might systematically fail for certain types of objects. E.g for types that are not extensible without copying the whole data, in particular for par::sys::heap, or for those where a possible strategy is not yet implemented.

Parameters:
addon The new amount of items of type T.

Definition at line 384 of file par_mem_chunk.h.

template<class T>
T* par::mem::chunk< T >::get ( void   )  const [inline]

template<class T>
T volatile* par::mem::chunk< T >::get_volatile ( void   )  const [inline]

Definition at line 481 of file par_mem_chunk.h.

template<class T>
T const* par::mem::chunk< T >::getro ( void   )  const [inline]

template<class T>
T volatile const* par::mem::chunk< T >::getro_volatile ( void   )  const [inline]

Definition at line 475 of file par_mem_chunk.h.

template<class T>
off_t par::mem::chunk< T >::Length ( void   )  const [inline]

template<class T>
void par::mem::chunk< T >::link ( char const *  URL  )  [inline]

Let this par::mem::chunk point to exactly the same whole object refered to by other.

If T has the const attribute, the object is accessed read-only. If in that case URL does not exist, it is not created but an empty par::mem::chunk is returned. Test for such a situation with Length().

Definition at line 253 of file par_mem_chunk.h.

template<class T>
T* par::mem::chunk< T >::map ( void   )  [inline]

template<class T>
T volatile* par::mem::chunk< T >::map_volatile ( void   )  [inline]

Definition at line 478 of file par_mem_chunk.h.

template<class T>
T const* par::mem::chunk< T >::mapro ( void   )  [inline]

template<class T>
T volatile const* par::mem::chunk< T >::mapro_volatile ( void   )  [inline]

Definition at line 472 of file par_mem_chunk.h.

template<class T>
off_t par::mem::chunk< T >::Offset (  )  const [inline]

Definition at line 393 of file par_mem_chunk.h.

template<class T >
par::mem::chunk< T >::operator chunk< T const > & ( void   )  [inline]

Const casting a chunk.

The weakness of C++ as a language make it necessary to define this operator. It ensure that a chunk< T > can be used everywhere a chunk< T const > is requested. E.g the copy operator chunk<T>& chunk<T>::operator<<(chunk< T const >&) needs a writable chunk on the left hand side but it may have a readonly or writable on the right.

Definition at line 734 of file par_mem_chunk.h.

template<class T >
par::mem::chunk< T >::operator chunk< T const >const & ( void   )  const [inline]

Definition at line 741 of file par_mem_chunk.h.

template<class T>
template<class oT >
bool par::mem::chunk< T >::operator!= ( chunk< oT > const &  other  )  const [inline]

The negation of both types of operator==.

Definition at line 522 of file par_mem_chunk.h.

template<class T>
chunk& par::mem::chunk< T >::operator<< ( par::mem::chunk< T const > &  there  )  [inline]

An alias for par::mem::chunk<T>::deepcopy().

Definition at line 544 of file par_mem_chunk.h.

template<class T>
chunk& par::mem::chunk< T >::operator= ( chunk< T > const &  other  )  [inline]

Reimplemented in par::step::array< T >.

Definition at line 259 of file par_mem_chunk.h.

template<class T>
template<class oT >
bool par::mem::chunk< T >::operator== ( chunk< oT > const &  other  )  const [inline]

Two chunks of different type will never be considered the same.

Definition at line 515 of file par_mem_chunk.h.

template<class T>
bool par::mem::chunk< T >::operator== ( chunk< T > const &  other  )  const [inline]

Two chunks over the same T type are the same if they point to the same heap and have same offset and length. No dynamic type checking is necessary.

Definition at line 507 of file par_mem_chunk.h.

template<class T>
void par::mem::chunk< T >::prefetch ( void   )  [inline]

Definition at line 395 of file par_mem_chunk.h.

template<class T>
par::mem::chunk< T >* par::mem::chunk< T >::split ( size_t  s,
off_t const *const   cnts,
par::mem::chunk< T > *  ret 
) const [inline]

Split chunk into chunks of given sizes.

Parameters:
s The number of chunks to split into.
cnts A vector of the s different sizes of these chunks.
ret A vector of chunks of size s. These chunks will be completely reininialized.

Definition at line 576 of file par_mem_chunk.h.

template<class T>
par::mem::chunk< T >* par::mem::chunk< T >::split ( size_t  s,
off_t const *const   cnts 
) const [inline]

Split chunk into chunks of given sizes.

Parameters:
s The number of chunks to split into.
cnts A vector of the s different sizes of these chunks.
Returns:
freshly allocated (with new) vector of chunks. Should be freed with delete[] before this chunk ceases existing.

Definition at line 562 of file par_mem_chunk.h.

template<class T>
static void par::mem::chunk< T >::static_init ( void   )  [static]

template<class T >
apointer< T > par::mem::chunk< T >::temp ( off_t  off  )  const [inline]

Definition at line 764 of file par_mem_chunk.h.

template<class T >
apointer< T > par::mem::chunk< T >::temp ( off_t  off,
off_t  len 
) const [inline]

Create a temporary to be used in a expression that expects a T* or similar.

If used in an expression that expects a T* the effect will be to map the chunk and provide the address of the data in question. Usually you should not use this to access individual items in the par::mem::chunk, since this would be very inefficient.

Definition at line 756 of file par_mem_chunk.h.

template<class T >
apointer< T > par::mem::chunk< T >::temp ( void   )  const [inline]

Definition at line 749 of file par_mem_chunk.h.

template<class T>
bool par::mem::chunk< T >::truncate ( off_t  nsize  )  throw (sys::error) [inline]

Change the size of the underlying object.

Only use this if you realy need to.

See also:
par::mem::req::truncate() for a description of the mechanism.
Precondition:
  • This chunk must have write access to the object.
  • The underlying object must be uniquely linked by this chunk.
  • The object must not be mapped.
  • The chunk must link to the entire object, i.e Offset() must be zero and Length() must be the length of the object.
Warning:
Any lock that might be held by somebody external to parXXL (eg. if the object is a file) may block this parXXL process.
Postcondition:
On success, a lock was obtained for the memory object. This lock is released before returning from this call.
Returns:
value of true means the call is on error.

Parameters:
nsize The new size of the object counted in items of type T.

Definition at line 374 of file par_mem_chunk.h.

template<class T>
void par::mem::chunk< T >::unmap ( void   )  [inline]

template<class T>
void par::mem::chunk< T >::unmapro ( void   )  [inline]

template<class T>
char const* par::mem::chunk< T >::url ( void   )  [inline]

Definition at line 592 of file par_mem_chunk.h.

template<class T>
chunk par::mem::chunk< T >::window ( char const *  URL,
off_t  off,
off_t  len 
) throw (sys::error) [inline]

Let this par::mem::chunk point to a window inside the object refered by URL.

If T has the const attribute, the object is accessed read-only. If in that case URL does not exist, it is not created but an empty par::mem::chunk is returned. Test for such a situation with Length().

Returns a reference to the calling object, to be compatible with par::mem::window.

Definition at line 226 of file par_mem_chunk.h.

template<class T>
chunk& par::mem::chunk< T >::window ( chunk< T const > const &  other,
off_t  off 
) [inline]

Definition at line 207 of file par_mem_chunk.h.

template<class T>
chunk& par::mem::chunk< T >::window ( chunk< T const > const &  other,
off_t  off,
off_t  len 
) [inline]

Let this par::mem::chunk point to a window inside the window of other.

Returns a reference to the calling object, to be compatible with par::mem::window.

Definition at line 199 of file par_mem_chunk.h.


Friends And Related Function Documentation

template<class T>
req::context* req::context::generate ( size_t  sessionId,
size_t  sessionSize,
size_t  mynum 
) [friend]


The documentation for this class was generated from the following file:

Generated on Tue Oct 13 22:04:17 2009 for parXXL by  doxygen 1.5.8