implementation of the DataHandlerBase class using ReadWriteCsv and Array2D. More...

#include <STK_DataHandler.h>

Inheritance diagram for STK::DataHandler:

Public Types
typedef DataHandlerBase< DataHandler >	Base

typedef DataHandlerBase< DataHandler >::InfoMap	InfoMap

Public Types inherited from STK::DataHandlerBase< DataHandler >
typedef std::map< std::string, std::string >	InfoMap

Public Member Functions
	DataHandler ()
	default constructor

	~DataHandler ()
	destructor

ReadWriteCsv const &	data () const
	get the whole data set

ReadWriteCsv const &	descriptor () const
	get the whole descriptor set

int	nbSample () const

int	nbVariable () const

void	setWithNames (bool withNames)
	set withNames flag

bool	readDataFromCsvFile (std::string const &datafile, std::string descriptorfile)
	read a data file and its companion description file.

bool	readDataFromCsvFile (std::string const &datafile, std::string const &idData, std::string const &idModel)
	read a data set from an Array2D.

template<typename Type >
bool	readDataFromArray2D (Array2D< Type > const &data, std::string const &idData, std::string const &idModel)
	read a data set from an Array2D.

template<typename Array >
bool	readDataFromArray (ExprBase< Array > const &data, std::string const &idData, std::string const &idModel)
	read a data set from an Array or Expression.

template<typename Type >
void	getData (std::string const &idData, Array2D< Type > &data) const

void	removeData (std::string const &idData)
	remove the data with the given idData

Public Member Functions inherited from STK::DataHandlerBase< DataHandler >
	~DataHandlerBase ()
	destructor

InfoMap const &	info () const

bool	addInfo (std::string const &idData, std::string const &idModel)
	Add an info descriptor to the data handler.

bool	getIdModelName (std::string const &idData, std::string &idModel) const
	Giving the Id of a data set, find the Id of the model.

void	writeInfo (ostream &os) const
	write infoMap on os

Public Member Functions inherited from STK::IRecursiveTemplate< Derived >
Derived &	asDerived ()
	static cast : return a reference of this with a cast to the derived class.

Derived const &	asDerived () const
	static cast : return a const reference of this with a cast to the derived class.

Derived *	asPtrDerived ()
	static cast : return a ptr on a `Derived` of this with a cast to the derived class.

Derived const *	asPtrDerived () const
	static cast : return a ptr on a constant `Derived` of this with a cast to the derived class.

Derived *	clone () const
	create a leaf using the copy constructor of the Derived class.

Derived *	clone (bool isRef) const
	create a leaf using the copy constructor of the Derived class and a flag determining if the clone is a reference or not.

Protected Member Functions
std::vector< int >	colIndex (std::string const &idData) const
	lookup on the descriptors in order to get the columns of the ReadWriteCsv with the Id idData.

Protected Member Functions inherited from STK::DataHandlerBase< DataHandler >
	DataHandlerBase ()
	default constructor

Protected Member Functions inherited from STK::IRecursiveTemplate< Derived >
	IRecursiveTemplate ()
	constructor.

	~IRecursiveTemplate ()
	destructor.

Private Attributes
bool	withNames_
	first line with names ?

ReadWriteCsv	data_
	data files

ReadWriteCsv	descriptor_
	descriptor files with two rows.

Additional Inherited Members
Protected Attributes inherited from STK::DataHandlerBase< DataHandler >
InfoMap	info_
	Store the informations of the mixtures in the form (idData, idModel) with.

Detailed Description

implementation of the DataHandlerBase class using ReadWriteCsv and Array2D.

The DataHandler class allow to read various csv files with their description files and to get the columns identified by an idData in an Array2D. All data are stored in memory in a ReadWriteCsv structure.

Definition at line 69 of file STK_DataHandler.h.

Member Typedef Documentation

◆ Base

typedef DataHandlerBase<DataHandler> STK::DataHandler::Base

Definition at line 72 of file STK_DataHandler.h.

◆ InfoMap

typedef DataHandlerBase<DataHandler>::InfoMap STK::DataHandler::InfoMap

Definition at line 73 of file STK_DataHandler.h.

Constructor & Destructor Documentation

◆ DataHandler()

STK::DataHandler::DataHandler ( )

inline

default constructor

Definition at line 75 of file STK_DataHandler.h.

75 : Base(), withNames_(false)

76 { data_.setWithNames(false); descriptor_.setWithNames(false);}

STK::DataHandler::descriptor_

ReadWriteCsv descriptor_

descriptor files with two rows.

Definition STK_DataHandler.h:142

STK::DataHandler::withNames_

bool withNames_

first line with names ?

Definition STK_DataHandler.h:136

STK::DataHandler::data_

ReadWriteCsv data_

data files

Definition STK_DataHandler.h:138

STK::DataHandler::Base

DataHandlerBase< DataHandler > Base

Definition STK_DataHandler.h:72

References data_, and descriptor_.

◆ ~DataHandler()

STK::DataHandler::~DataHandler ( )

inline

destructor

Definition at line 78 of file STK_DataHandler.h.

78{}

Member Function Documentation

◆ colIndex()

std::vector< int > STK::DataHandler::colIndex ( std::string const & idData ) const

protected

lookup on the descriptors in order to get the columns of the ReadWriteCsv with the Id idData.

Parameters

idData id of the data to get

Definition at line 138 of file STK_DataHandler.cpp.

{
  int rowIdData = descriptor_.beginRows()+1;
  std::vector<int> colindex;
  for (int i = descriptor_.beginCols(); i <= descriptor_.lastIdxCols(); ++i)
  { if (descriptor_.var(i).at(rowIdData) == idData) colindex.push_back(i);}
  return colindex;
}

References descriptor_.

Referenced by getData().

◆ data()

ReadWriteCsv const & STK::DataHandler::data ( ) const

inline

get the whole data set

Definition at line 80 of file STK_DataHandler.h.

80{ return data_;}

References data_.

Referenced by getData(), readDataFromArray(), readDataFromArray2D(), and readDataFromCsvFile().

◆ descriptor()

ReadWriteCsv const & STK::DataHandler::descriptor ( ) const

inline

get the whole descriptor set

Definition at line 82 of file STK_DataHandler.h.

82{ return descriptor_;}

References descriptor_.

◆ getData()

template<typename Type >

void STK::DataHandler::getData	(	std::string const &	idData,
		Array2D< Type > &	data
	)		const

Returns: in an Array2D<Type> the data with the given idData

Definition at line 148 of file STK_DataHandler.h.

{
  std::vector<int> indexes = colIndex(idData);
#ifdef STK_MIXTURE_VERY_VERBOSE
  stk_cout << _T("In DataHandler::getData, idData = ") << idData << _T("\n");
  stk_cout << _T("columns found = ");
  for (std::vector<int>::const_iterator it = indexes.begin(); it != indexes.end(); ++it)
  { stk_cout << (*it) << _T(" ");}
  stk_cout << _T("\n");
#endif
  int nbVariable = indexes.size();
  data.resize(nbSample(), nbVariable);
  int j= data.beginCols();
  for (std::vector<int>::const_iterator it = indexes.begin(); it != indexes.end(); ++it, ++j)
  {
    for (int i = data_.firstRow(*it); i <= data_.lastRow(*it); ++i)
    { data(i, j) = stringToType<Type>(data_(i,*it));}
  }
}

References _T, colIndex(), data(), data_, nbSample(), nbVariable(), and stk_cout.

◆ nbSample()

int STK::DataHandler::nbSample ( ) const

inline

Returns: the number of sample (the number of rows of the data)

Definition at line 84 of file STK_DataHandler.h.

84{ return data_.sizeRows();}

References data_.

Referenced by getData().

◆ nbVariable()

int STK::DataHandler::nbVariable ( ) const

inline

Returns: the number of sample (the number of columns of the data)

Definition at line 86 of file STK_DataHandler.h.

86{ return data_.size();}

References data_.

Referenced by getData().

◆ readDataFromArray()

template<typename Array >

bool STK::DataHandler::readDataFromArray	(	ExprBase< Array > const &	data,
		std::string const &	idData,
		std::string const &	idModel
	)

read a data set from an Array or Expression.

This method should be essentially used:

for testing some statistical method as the data will be converted in a String format (whcih is not an efficient way to store the data..).
if the data are already stored in a String format.
Parameters

data the data set

idData the id of the data

idModel an id identifying the model to use with the data set

Definition at line 191 of file STK_DataHandler.h.

{
  // add descriptor
  Variable<std::string> desc(2, stringNa);
  desc[baseIdx] = idModel ; desc[baseIdx+1] = idData;
  if (!addInfo(idData, idModel)) return false;
  // store data at the end of the ReadWriteCsv array in a string format
  for (int j=data.beginCols(); j<= data.lastIdxCols(); ++j)
  {
    data_.push_back();
    data_.back().resize(data.rows());
    for (int i= data.beginRows(); i < data.endRows(); ++i)
    { data_.back()[i] = typeToString(data.elt(i,j), std::scientific);}
    // store descriptor : this is the same for all the columns added
    descriptor_.push_back(desc);
  }
  return true;
}

References STK::DataHandlerBase< DataHandler >::addInfo(), STK::baseIdx, data(), data_, descriptor_, STK::stringNa, and STK::typeToString().

◆ readDataFromArray2D()

template<typename Type >

bool STK::DataHandler::readDataFromArray2D	(	Array2D< Type > const &	data,
		std::string const &	idData,
		std::string const &	idModel
	)

read a data set from an Array2D.

This method should be essentially used:

for testing some statistical method as the data will be converted in a String format (whcih is not an efficient way to store the data..).
if the data are already stored in a String format.
Parameters

data the data set

idData the id of the data

idModel an id identifying the model to use with the data set

Definition at line 169 of file STK_DataHandler.h.

{
  // add descriptor
  Variable<std::string> desc(2, stringNa);
  desc[baseIdx] = idModel ; desc[baseIdx+1] = idData;
  if (!addInfo(idData, idModel)) return false;
  // store data at the end of the ReadWriteCsv array in a string format
  for (int j=data.beginCols(); j<= data.lastIdxCols(); ++j)
  {
    data_.push_back();
    data_.back().resize(data.rows());
    for (int i= data.beginRows(); i < data.endRows(); ++i)
    { data_.back()[i] = typeToString(data(i,j), std::scientific);}
    // store descriptor : this is the same for all the columns added
    descriptor_.push_back(desc);
  }
  return true;
}

References STK::DataHandlerBase< DataHandler >::addInfo(), STK::baseIdx, data(), data_, descriptor_, STK::stringNa, and STK::typeToString().

◆ readDataFromCsvFile() [1/2]

bool STK::DataHandler::readDataFromCsvFile	(	std::string const &	datafile,
		std::string const &	idData,
		std::string const &	idModel
	)

read a data set from an Array2D.

read a data file

Parameters

datafile	the data file to get
idData	the id of the data
idModel	an id identifying the model to use with the data set

Definition at line 41 of file STK_DataHandler.cpp.

{
  ReadWriteCsv data(datafile);
  // no names to read in the first line
  data.setWithNames(withNames_);
  // read the data set
  if (!data.read())
  {
    stk_cerr << _T("An error occur when reading the data file.\nWhat: ")
             << data.error();
    return false;
  }
  // add descriptor
  Variable<std::string> desc(2, stringNa);
  desc[baseIdx] = idModel ; desc[baseIdx+1] = idData;
  // store data and descriptors
  if (!addInfo(idData, idModel)) return false;
  data_ += data;
  // store descriptor : this is the same for all the columns added
  for (int j=data.beginCols(); j< data.endCols(); ++j)
  { descriptor_.push_back(desc);}
  return true;
}

References _T, STK::DataHandlerBase< DataHandler >::addInfo(), STK::baseIdx, data(), data_, descriptor_, STK::IRunnerBase::error(), stk_cerr, STK::stringNa, and withNames_.

◆ readDataFromCsvFile() [2/2]

bool STK::DataHandler::readDataFromCsvFile	(	std::string const &	datafile,
		std::string	descriptorfile
	)

read a data file and its companion description file.

Definition at line 68 of file STK_DataHandler.cpp.

{
  ReadWriteCsv rwdata(datafile);
  // no names to read in the first line
  rwdata.setWithNames(withNames_);
  // read the data set
  if (!rwdata.read())
  {
    stk_cerr << _T("An error occur when reading the data file.\nWhat: ")
             << rwdata.error();
    return false;
  }
  ReadWriteCsv rwdesc(descriptorfile);
  // no names to read in the first line
  rwdesc.setWithNames(false);
  // read the data set
  if (!rwdesc.read())
  {
    stk_cerr << _T("An error occur when reading the descriptor file.\nWhat: ")
             << rwdesc.error();
    return false;
  }
  // check logic
  if (rwdata.size() != rwdesc.size())
  {
    stk_cerr << _T("Data file and descriptor file does not have the same number of column.\n");
    return false;
  }
  if (rwdata.sizeRows() == 0)
  {
    stk_cerr << _T("No data.\n");
    return false;
  }
  if (rwdesc.sizeRows() < 2)
  {
    stk_cerr << _T("No descriptor.\n");
    return false;
  }
  // parse descriptor file
  int firstRow = rwdesc.beginRows();
  for (int j=rwdesc.beginCols(); j< rwdesc.endCols(); j++)
  {
    std::string idModel = rwdesc.at(j).at(firstRow);
    std::string idData = rwdesc.at(j).at(firstRow+1);
    if (!addInfo(idData, idModel)) return false;
  }
  // store data and descriptors
  data_       += rwdata;
  descriptor_ += rwdesc;
  return true;
}

References _T, STK::DataHandlerBase< DataHandler >::addInfo(), data_, descriptor_, STK::IRunnerBase::error(), stk_cerr, and withNames_.

◆ removeData()

void STK::DataHandler::removeData ( std::string const & idData )

remove the data with the given idData

Definition at line 121 of file STK_DataHandler.cpp.

{
  int rowIdData = descriptor_.beginRows()+1;
  for (int i = descriptor_.endCols()-1; i >= descriptor_.beginCols(); --i)
  { if (descriptor_.var(i)[rowIdData] == idData)
    {
      data_.eraseColumn(i);
      descriptor_.eraseColumn(i);
     }
  }
  info_.erase(idData);
}

References data_, descriptor_, and STK::DataHandlerBase< DataHandler >::info_.

◆ setWithNames()

void STK::DataHandler::setWithNames ( bool withNames )

inline

set withNames flag

Definition at line 89 of file STK_DataHandler.h.

89{ withNames_ = withNames;}

References withNames_.

Member Data Documentation

◆ data_

ReadWriteCsv STK::DataHandler::data_

private

data files

Definition at line 138 of file STK_DataHandler.h.

Referenced by data(), DataHandler(), getData(), nbSample(), nbVariable(), readDataFromArray(), readDataFromArray2D(), readDataFromCsvFile(), readDataFromCsvFile(), and removeData().

◆ descriptor_

ReadWriteCsv STK::DataHandler::descriptor_

private

descriptor files with two rows.

On the first row we get the idModel, on the second row, we get the idData

Definition at line 142 of file STK_DataHandler.h.

Referenced by colIndex(), DataHandler(), descriptor(), readDataFromArray(), readDataFromArray2D(), readDataFromCsvFile(), readDataFromCsvFile(), and removeData().

◆ withNames_

bool STK::DataHandler::withNames_

private

first line with names ?

Definition at line 136 of file STK_DataHandler.h.

Referenced by readDataFromCsvFile(), readDataFromCsvFile(), and setWithNames().

The documentation for this class was generated from the following files:

data	the data set
idData	the id of the data
idModel	an id identifying the model to use with the data set

Public Types

Public Member Functions

Protected Member Functions

Private Attributes

Additional Inherited Members

Detailed Description

Member Typedef Documentation

◆ Base

◆ InfoMap

Constructor & Destructor Documentation

◆ DataHandler()

◆ ~DataHandler()

Member Function Documentation

◆ colIndex()

◆ data()

◆ descriptor()

◆ getData()

◆ nbSample()

◆ nbVariable()

◆ readDataFromArray()

◆ readDataFromArray2D()

◆ readDataFromCsvFile() [1/2]

◆ readDataFromCsvFile() [2/2]

◆ removeData()

◆ setWithNames()

Member Data Documentation

◆ data_

◆ descriptor_

◆ withNames_