ug4
|
Performs communication between interfaces on different processes. More...
#include <pcl_interface_communicator.h>
Classes | |
struct | ExtractorInfo |
holds information that will be passed to the extract routines. More... | |
Public Types | |
typedef Layout::Interface | Interface |
typedef TLayout | Layout |
typedef Layout::Type | Type |
Public Member Functions | |
bool | communicate (int tag=749345) |
sends and receives the collected data. More... | |
bool | communicate_and_resume (int tag=749345) |
collects data and communicates it with other processes without waiting for receive More... | |
bool | communication_debugging_enabled () |
returns true if communication debugging is enabled More... | |
void | disable_communication_debugging () |
disables debugging of communication More... | |
void | enable_communication_debugging (const ProcessCommunicator &involvedProcs=ProcessCommunicator(PCD_WORLD)) |
enables debugging of communication. This has a severe effect on performance! More... | |
template<class TLayoutMap > | |
void | exchange_data (const TLayoutMap &layoutMap, const typename TLayoutMap::Key &keyFrom, const typename TLayoutMap::Key &keyTo, ICommunicationPolicy< TLayout > &commPol) |
internally calls send_data and receive_data with the specified layouts. More... | |
InterfaceCommunicator () | |
void | receive_data (const Layout &layout, ICommunicationPolicy< TLayout > &commPol) |
registers an communication-policy to receive data on communicate. More... | |
void | receive_data (int srcProc, const Interface &interface, ICommunicationPolicy< TLayout > &commPol) |
registers a communication-policy to receive data on communicate. More... | |
void | receive_raw (int srcProc, ug::BinaryBuffer &bufOut, int bufSize=-1) |
registers a binary-stream to receive data from a source-proc. More... | |
void | receive_raw (int srcProc, void *bufOut, int bufSize) |
registers a buffer to receive data from a source-proc. More... | |
void | send_data (const Layout &layout, ICommunicationPolicy< TLayout > &commPol) |
collects data that will be send during communicate. More... | |
void | send_data (int targetProc, const Interface &interface, ICommunicationPolicy< TLayout > &commPol) |
collects data that will be send during communicate. More... | |
void | send_raw (int targetProc, const void *pBuff, int bufferSize, bool bSizeKnownAtTarget=false) |
sends raw data to a target-proc. More... | |
void | wait () |
waits for the data communicated by communicate_and_resume() and extracts it More... | |
Protected Types | |
typedef std::map< int, ug::BinaryBuffer > | BufferMap |
typedef ICommunicationPolicy< Layout > | CommPol |
typedef std::list< ExtractorInfo > | ExtractorInfoList |
A list that holds information about extractors. More... | |
Protected Member Functions | |
bool | collect_layout_buffer_sizes (const TLayout &layout, ICommunicationPolicy< TLayout > &commPol, std::map< int, int > *pMapBuffSizesOut, const layout_tags::multi_level_layout_tag &) |
collects buffer sizes for a given layout and stores them in a map More... | |
bool | collect_layout_buffer_sizes (const TLayout &layout, ICommunicationPolicy< TLayout > &commPol, std::map< int, int > *pMapBuffSizesOut, const layout_tags::single_level_layout_tag &) |
collects buffer sizes for a given layout and stores them in a map More... | |
void | extract_data (const TLayout &layout, BufferMap &bufMap, CommPol &extractor) |
extract data from stream-pack More... | |
void | extract_data (const TLayout &layout, BufferMap &bufMap, CommPol &extractor, const layout_tags::multi_level_layout_tag &) |
void | extract_data (const TLayout &layout, BufferMap &bufMap, CommPol &extractor, const layout_tags::single_level_layout_tag &) |
void | prepare_receiver_buffer_map (BufferMap &bufMap, std::set< int > &curProcs, const TLayout &layout) |
prepare stream-pack-in More... | |
void | prepare_receiver_buffer_map (BufferMap &streamPack, std::set< int > &curProcs, const TLayout &layout, const layout_tags::multi_level_layout_tag &) |
specialization of stream-pack preparation for multi-level-layouts More... | |
void | prepare_receiver_buffer_map (BufferMap &streamPack, std::set< int > &curProcs, const TLayout &layout, const layout_tags::single_level_layout_tag &) |
specialization of stream-pack preparation for single-level-layouts More... | |
void | send_data (const Layout &layout, ICommunicationPolicy< TLayout > &commPol, const layout_tags::multi_level_layout_tag &) |
helper to collect data from multi-level-layouts More... | |
void | send_data (const Layout &layout, ICommunicationPolicy< TLayout > &commPol, const layout_tags::single_level_layout_tag &) |
helper to collect data from single-level-layouts More... | |
Protected Attributes | |
bool | m_bDebugCommunication |
true if the communication shall be debugged. More... | |
bool | m_bSendBuffersFixed |
holds info whether all send-buffers are of predetermined fixed size. More... | |
BufferMap | m_bufMapIn |
holds the buffers that are used to receive data More... | |
BufferMap | m_bufMapOut |
holds the buffers that are used to send data More... | |
int | m_curComTag |
This is the tag for the currently performed communication. More... | |
std::set< int > | m_curInProcs |
stores in-procs for the next communication step More... | |
std::set< int > | m_curOutProcs |
stores out-procs for the next communication step More... | |
ProcessCommunicator | m_debugProcComm |
This procComm holds the processes that shall participate during communication-debugging. More... | |
ExtractorInfoList | m_extractorInfos |
holds information about the extractors that are awaiting data. More... | |
std::vector< MPI_Request > | m_vReceiveRequests |
std::vector< MPI_Request > | m_vSendRequests |
used by communicate, communicate_and_resume and wait, to check whether communication is done. More... | |
Performs communication between interfaces on different processes.
|
protected |
|
protected |
|
protected |
A list that holds information about extractors.
typedef Layout::Interface pcl::InterfaceCommunicator< TLayout >::Interface |
typedef TLayout pcl::InterfaceCommunicator< TLayout >::Layout |
typedef Layout::Type pcl::InterfaceCommunicator< TLayout >::Type |
pcl::InterfaceCommunicator< TLayout >::InterfaceCommunicator |
|
protected |
collects buffer sizes for a given layout and stores them in a map
The given map holds pairs of procID, bufferSize If buffer-sizes can't be determined, false is returned. if pMmapBuffSizesOut == NULL, the method will simply determine whether all buffersizes can be calculated.
|
protected |
collects buffer sizes for a given layout and stores them in a map
The given map holds pairs of procID, bufferSize If buffer-sizes can't be determined, false is returned. if pMmapBuffSizesOut == NULL, the method will simply determine whether all buffersizes can be calculated.
bool pcl::InterfaceCommunicator< TLayout >::communicate | ( | int | tag = 749345 | ) |
sends and receives the collected data.
The collected data will be send to the associated processes. The extract routines of the communication-policies which were registered through Communicator::receive_data will be called with the received data. After all received data is processed, the communication-policies are released. Make sure that you will keep your communication-policies in memory until this point.
tag | For internal communications the provided tag is used. The default value is fine in most cases and normally only has to be adjusted if one performs multiple communications at the same time (e.g. with different communicators). |
Referenced by ug::AddConnectionsBetweenSlaves(), ug::AddExtraProcessEntriesToSubdomainLayout(), ug::AdditiveToConsistent(), ug::AdditiveToUnique(), ug::SurfaceView::adjust_parallel_surface_states(), ug::ApplySmoothManifoldPosToTopLevelAveragingScheme(), ug::ApplySmoothManifoldPosToTopLevelButterflyScheme(), ug::ApplySmoothManifoldPosToTopLevelLoopScheme(), ug::ApplySmoothVolumePosToTopLevel(), ug::AttachmentAllReduce(), ug::BroadcastVectorFromOne(), ug::BuildDomainDecompositionLayouts(), ug::BuildOneToManyLayout(), ug::CalculateNumElemsVertexAttachmentInTopLevel(), ug::CalculateNumManifoldEdgesVertexAttachmentInParentLevel(), ug::CalculateNumManifoldFacesVertexAttachmentInTopLevel(), ug::ParallelVector< TVector >::check_storage_type(), ug::CheckDistributedObjectConstraintTypes(), ug::ParallelColoring::color(), ug::CommunicateConnections(), ug::CommunicateOnInterfaces(), ug::CopyValues(), ug::CreateAndDistributeGlobalIDs(), ug::CreateSurfaceView(), ug::ComPol_CheckDistributedParentStates< TLayout >::exchange_data(), ug::ComPol_NewConstrainedVerticals< TLayout >::exchange_data(), ug::ExchangeAndAdjustSideErrors(), ug::GatherVectorOnOne(), ug::ParallelDualGraph< TGeomBaseObj, TIndexType, TConnectingObj >::generate_graph(), ug::GenerateGlobalAlgebraIDs(), ug::GenerateGlobalConsecutiveIndices(), ug::GridFunction< TDomain, TAlgebra >::grid_changed_callback(), ug::GridFunction< TDomain, TAlgebra >::grid_distribution_callback(), ug::MarkForAdaption_GradientAverage(), ug::MatAddSlaveRowsToMasterRowOverlap0(), ug::MatMakeConsistentOverlap0(), ug::PartitionMultiGridLevel_ParmetisKway(), ug::ProjectHierarchyToSubdivisionLimit(), ug::CopyAttachmentHandler< TElem, TAttachment >::propagate_to_level(), ug::ReceiveGlobalLayout(), ug::ReceiveMatrix(), ug::ReceiveOnInterfaces(), ug::SendGlobalLayout(), ug::SendMatrix(), ug::SendOnInterfaces(), ug::SideAndElemErrEstData< TDomain >::summarize_err_est_data(), ug::SynchronizeDistInfos(), pcl::TestLayoutIsDoubleEnded(), pcl::TestSizeOfInterfacesInLayoutsMatch(), ug::UniqueToConsistent(), ug::VecBroadcast(), ug::VecCopy(), ug::VecGather(), ug::VecSubtractOneSlaveFromMaster(), and ug::VecSubtractOnLayout().
bool pcl::InterfaceCommunicator< TLayout >::communicate_and_resume | ( | int | tag = 749345 | ) |
collects data and communicates it with other processes without waiting for receive
Data will be collected from the communication-policies registered in send(...) and communicated to other processes. The method however won't wait until the data has arrived at its target processes and thus can't extract the received data. This has to be done manually by calling wait().
tag | For internal communications the provided tag is used. The default value is fine in most cases and normally only has to be adjusted if one performs multiple communications at the same time (e.g. with different communicators). |
References ug::BinaryBuffer::buffer(), pcl::ICommunicationPolicy< TLayout >::get_required_buffer_size(), ug::LIB_PCL, pcl::InterfaceCommunicator< TLayout >::ExtractorInfo::m_extractor, pcl::InterfaceCommunicator< TLayout >::ExtractorInfo::m_interface, pcl::InterfaceCommunicator< TLayout >::ExtractorInfo::m_layout, pcl::InterfaceCommunicator< TLayout >::ExtractorInfo::m_rawSize, pcl::InterfaceCommunicator< TLayout >::ExtractorInfo::m_srcProc, PCL_COMM_WORLD, PCL_PROFILE, ug::BinaryBuffer::reserve(), pcl::SendRecvBuffersMatch(), pcl::SendRecvListsMatch(), ug::BinaryBuffer::set_write_pos(), UG_DLOG, UG_LOG, UG_THROW, pcl::Waitall(), and ug::BinaryBuffer::write_pos().
bool pcl::InterfaceCommunicator< TLayout >::communication_debugging_enabled |
returns true if communication debugging is enabled
void pcl::InterfaceCommunicator< TLayout >::disable_communication_debugging |
disables debugging of communication
void pcl::InterfaceCommunicator< TLayout >::enable_communication_debugging | ( | const ProcessCommunicator & | involvedProcs = ProcessCommunicator(PCD_WORLD) | ) |
enables debugging of communication. This has a severe effect on performance!
communication debugging will execute some code during communicate(), which checks whether matching sends and receives have been scheduled with matching buffer sizes.
If not all processes participate during communication, you have to specify a process-communicator (involvedProcs), which includes all and only the procs, which will call communicate later on. If no process communicator is specified, then PCD_WORLD is used.
Note that communication debugging introduces additional communication which considerable slows down performance. You should only use it temporarily, if you encounter problems during communication.
Don't forget to call disable_communication_debugging(), when you're done with debugging.
References UG_LOG.
void pcl::InterfaceCommunicator< TLayout >::exchange_data | ( | const TLayoutMap & | layoutMap, |
const typename TLayoutMap::Key & | keyFrom, | ||
const typename TLayoutMap::Key & | keyTo, | ||
ICommunicationPolicy< TLayout > & | commPol | ||
) |
internally calls send_data and receive_data with the specified layouts.
Note that data is not communicated until communicate has been called.
TLayout has to feature the following typedefs and methods:
The methods will only be called with type InterfaceCommunicator::Type.
This method is particularily useful if you categorize layouts on a process. If you separate your layouts into master and slave layouts, you could use this method e.g. to copy data from all master-layouts to all slave-layouts of a type with a single call.
Referenced by ug::SurfaceView::adjust_parallel_surface_states(), ug::ApplySmoothManifoldPosToTopLevelAveragingScheme(), ug::ApplySmoothManifoldPosToTopLevelButterflyScheme(), ug::ApplySmoothManifoldPosToTopLevelLoopScheme(), ug::ApplySmoothVolumePosToTopLevel(), ug::AttachmentAllReduce(), ug::CalculateNumElemsVertexAttachmentInTopLevel(), ug::CalculateNumManifoldEdgesVertexAttachmentInParentLevel(), ug::CalculateNumManifoldFacesVertexAttachmentInTopLevel(), ug::CheckDistributedObjectConstraintTypes(), ug::CreateAndDistributeGlobalIDs(), ug::ComPol_NewConstrainedVerticals< TLayout >::exchange_data(), ug::ExchangeAndAdjustSideErrors(), ug::GridFunction< TDomain, TAlgebra >::grid_changed_callback(), ug::GridFunction< TDomain, TAlgebra >::grid_distribution_callback(), ug::MarkForAdaption_GradientAverage(), ug::ProjectHierarchyToSubdivisionLimit(), ug::CopyAttachmentHandler< TElem, TAttachment >::propagate_to_level(), ug::SideAndElemErrEstData< TDomain >::summarize_err_est_data(), and ug::SynchronizeDistInfos().
|
protected |
extract data from stream-pack
References PCL_PROFILE_FUNC.
|
protected |
|
protected |
|
protected |
prepare stream-pack-in
|
protected |
specialization of stream-pack preparation for multi-level-layouts
|
protected |
specialization of stream-pack preparation for single-level-layouts
void pcl::InterfaceCommunicator< TLayout >::receive_data | ( | const Layout & | layout, |
ICommunicationPolicy< TLayout > & | commPol | ||
) |
registers an communication-policy to receive data on communicate.
Receives have to be registered before communicate is executed. make sure that your instance of the communication-policy exists until communicate has benn executed.
void pcl::InterfaceCommunicator< TLayout >::receive_data | ( | int | srcProc, |
const Interface & | interface, | ||
ICommunicationPolicy< TLayout > & | commPol | ||
) |
registers a communication-policy to receive data on communicate.
Receives have to be registered before communicate is executed. make sure that your instance of the communication-policy exists until communicate has benn executed.
Referenced by ug::AddExtraProcessEntriesToSubdomainLayout(), ug::AdditiveToConsistent(), ug::AdditiveToUnique(), ug::BroadcastVectorFromOne(), ug::BuildDomainDecompositionLayouts(), ug::BuildOneToManyLayout(), ug::ParallelVector< TVector >::check_storage_type(), ug::CommunicateConnections(), ug::CopyValues(), ug::CreateSurfaceView(), ug::ComPol_CheckDistributedParentStates< TLayout >::exchange_data(), ug::GatherVectorOnOne(), ug::ParallelDualGraph< TGeomBaseObj, TIndexType, TConnectingObj >::generate_graph(), ug::GenerateGlobalAlgebraIDs(), ug::GenerateGlobalConsecutiveIndices(), ug::MatAddSlaveRowsToMasterRowOverlap0(), ug::MatMakeConsistentOverlap0(), ug::PartitionMultiGridLevel_ParmetisKway(), ug::UniqueToConsistent(), ug::VecBroadcast(), ug::VecCopy(), ug::VecGather(), ug::VecSubtractOneSlaveFromMaster(), and ug::VecSubtractOnLayout().
void pcl::InterfaceCommunicator< TLayout >::receive_raw | ( | int | srcProc, |
ug::BinaryBuffer & | bufOut, | ||
int | bufSize = -1 |
||
) |
registers a binary-stream to receive data from a source-proc.
Receives data that has been sent with send_raw. If send_raw was called with bSizeKnownAtTarget == true, an exact bufferSize has to be specified. If bSizeKnownAtTarget was set to false, bufferSize has to be set to -1.
Make sure that binStreamOut exists until communicate has been executed.
Please note that this method should only be used if custom data should be send in a block with data that is communicated through interfaces, since an additional copy-operation has to be performed. If you're only interested in sending raw data, you should take a look into pcl::ProcessCommunicator::receive.
Referenced by ug::AddConnectionsBetweenSlaves(), ug::ParallelColoring::color(), ug::CommunicateOnInterfaces(), ug::NewLayoutCreator::issue(), ug::ParallelNodes::issue(), ug::RowSendingScheme< matrix_type >::issue_send(), ug::ReceiveGlobalLayout(), ug::ReceiveMatrix(), ug::ReceiveOnInterfaces(), pcl::TestLayoutIsDoubleEnded(), and pcl::TestSizeOfInterfacesInLayoutsMatch().
void pcl::InterfaceCommunicator< TLayout >::receive_raw | ( | int | srcProc, |
void * | bufOut, | ||
int | bufSize | ||
) |
registers a buffer to receive data from a source-proc.
Receives data that has been sent with send_raw. This method may only be used if send_raw was called with bSizeKnownAtTarget == true. Call receive_raw with a binary- stream instead, if buffer-sizes are not known.
Make sure that the buffer points to valid memory until communicate has been executed.
Please note that this method should only be used if custom data should be send in a block with data that is communicated through interfaces, since an additional copy-operation has to be performed. If you're only interested in sending raw data, you should take a look into pcl::ProcessCommunicator::receive.
void pcl::InterfaceCommunicator< TLayout >::send_data | ( | const Layout & | layout, |
ICommunicationPolicy< TLayout > & | commPol | ||
) |
collects data that will be send during communicate.
Calls ICommunicationPolicy<TLayout>::collect with the specified layout and the binary stream that is associated with the layouts target processes. Note that data will not be send until communicate has been called.
References PCL_PROFILE.
|
protected |
helper to collect data from multi-level-layouts
|
protected |
helper to collect data from single-level-layouts
void pcl::InterfaceCommunicator< TLayout >::send_data | ( | int | targetProc, |
const Interface & | interface, | ||
ICommunicationPolicy< TLayout > & | commPol | ||
) |
collects data that will be send during communicate.
Calls ICommunicationPolicy<TLayout>::collect with the specified interface and the binary stream that is associated with the specified target process. Note that data will not be send until communicate has been called.
Referenced by ug::AddExtraProcessEntriesToSubdomainLayout(), ug::AdditiveToConsistent(), ug::AdditiveToUnique(), ug::BroadcastVectorFromOne(), ug::BuildDomainDecompositionLayouts(), ug::BuildOneToManyLayout(), ug::ParallelVector< TVector >::check_storage_type(), ug::CommunicateConnections(), ug::CopyValues(), ug::CreateSurfaceView(), ug::ComPol_CheckDistributedParentStates< TLayout >::exchange_data(), ug::GatherVectorOnOne(), ug::ParallelDualGraph< TGeomBaseObj, TIndexType, TConnectingObj >::generate_graph(), ug::GenerateGlobalAlgebraIDs(), ug::GenerateGlobalConsecutiveIndices(), ug::MatAddSlaveRowsToMasterRowOverlap0(), ug::MatMakeConsistentOverlap0(), ug::PartitionMultiGridLevel_ParmetisKway(), ug::UniqueToConsistent(), ug::VecBroadcast(), ug::VecCopy(), ug::VecGather(), ug::VecSubtractOneSlaveFromMaster(), and ug::VecSubtractOnLayout().
void pcl::InterfaceCommunicator< TLayout >::send_raw | ( | int | targetProc, |
const void * | pBuff, | ||
int | bufferSize, | ||
bool | bSizeKnownAtTarget = false |
||
) |
sends raw data to a target-proc.
Shedules the data in pBuff to be sent to the target-proc pBuff can be reused or cleared directly after the call returns.
Data sent with this method can be received using receive_raw.
Please note that this method should only be used if custom data should be send in a block with data that is communicated through interfaces, since an additional copy-operation at the target process has to be performed. If you're only interested in sending raw data, you should take a look into pcl::ProcessCommunicator::send.
References pcl::NumProcs(), and ug::BinaryBuffer::write().
Referenced by ug::AddConnectionsBetweenSlaves(), ug::ParallelColoring::color(), ug::CommunicateOnInterfaces(), ug::NewLayoutCreator::issue(), ug::ParallelNodes::issue(), ug::RowSendingScheme< matrix_type >::issue_send(), ug::SendGlobalLayout(), ug::SendMatrix(), ug::SendOnInterfaces(), pcl::TestLayoutIsDoubleEnded(), and pcl::TestSizeOfInterfacesInLayoutsMatch().
void pcl::InterfaceCommunicator< TLayout >::wait |
waits for the data communicated by communicate_and_resume() and extracts it
The extract routines of the communication-policies which were registered through Communicator::receive_data will be called with the received data. After all received data is processed, the communication-policies are released. Make sure that you will keep your communication-policies in memory until this point.
References ug::BinaryBuffer::buffer(), ug::BinaryBuffer::clear(), pcl::ICommunicationPolicy< TLayout >::extract(), pcl::InterfaceCommunicator< TLayout >::ExtractorInfo::m_binBuffer, pcl::InterfaceCommunicator< TLayout >::ExtractorInfo::m_buffer, pcl::InterfaceCommunicator< TLayout >::ExtractorInfo::m_extractor, pcl::InterfaceCommunicator< TLayout >::ExtractorInfo::m_interface, pcl::InterfaceCommunicator< TLayout >::ExtractorInfo::m_layout, pcl::InterfaceCommunicator< TLayout >::ExtractorInfo::m_rawSize, pcl::InterfaceCommunicator< TLayout >::ExtractorInfo::m_srcProc, PCL_PROFILE, ug::BinaryBuffer::read(), ug::BinaryBuffer::reserve(), ug::BinaryBuffer::set_write_pos(), and pcl::Waitall().
|
protected |
true if the communication shall be debugged.
|
protected |
holds info whether all send-buffers are of predetermined fixed size.
reset to true after each communication-step.
|
protected |
holds the buffers that are used to receive data
|
protected |
holds the buffers that are used to send data
|
protected |
This is the tag for the currently performed communication.
Set to -1 if no communication is currently performed.
|
protected |
stores in-procs for the next communication step
|
protected |
stores out-procs for the next communication step
|
protected |
This procComm holds the processes that shall participate during communication-debugging.
|
protected |
holds information about the extractors that are awaiting data.
|
protected |
|
protected |
used by communicate, communicate_and_resume and wait, to check whether communication is done.