• Do not register here on develop.twiki.org, login with your twiki.org account.
• Use View topic Item7848 for generic doc work for TWiki-6.1.1. Use View topic Item7851 for doc work on extensions that are not part of a release. More... Close
• Anything you create or change in standard webs (Main, TWiki, Sandbox etc) will be automatically reverted on every SVN update.
Does this site look broken?. Use the LitterTray web for test cases.
TWiki::Data::DelimitedFile has functionality that should be usable throughout TWiki and its plugins. However, it is not well documented nor is it exposed. Sooner or later someone will end up wasting their time duplicating the functionality because they need it and don't know it's already been implemented. This will be bad because then we'll have wasted time and have incompatible versions.

It will read from a file into a tabular data structure or save a file from the tabular data structure containing delimited (e.g. tab, comma or pipe) entries, ensuring that the lines read conform to the heading row.

sub read {
    my (%settings) = @_;
    my $filename = $settings{filename};
    my $content = $settings{content};
    my $delimiter = $settings{delimiter} || "\t";
    my $rowstart = $settings{rowstart} || '';
    my $rowend = $settings{rowend} || "\n";
    return (\@fieldNames, %data);

sub save {
    my (%settings) = @_;
    my $filename = $settings{filename} ||
      throw Error::Simple( 'Save: filename parameter is mandatory' );
    my $delimiter = $settings{delimiter} || "\t";
    my $rowstart = $settings{rowstart} || '';
    my $rowend = $settings{rowend} || "\n";
    my %data = %{$settings{data} ||
                   throw Error::Simple( 'Save: data parameter is mandatory' )};
    my @fieldNames = @{$settings{fieldNames} ||
                         throw Error::Simple( 'Save: fieldNames parameter is man
datory' )};

There is no testsuite for this module but it is exercised as a consequence of testing bulk register in the RegisterTests.pm

There appears to be an adverse call to Data::Dumper in save() - perhaps this has not been reported as an error because only read is used? I don't recall.

-- MC

Crawford kindly factored out TWiki::Data::DelimitedFile during http://develop.twiki.org/trac/changeset?new=twiki%2Fbranches%2FMAIN%2Flib%2FTWiki%2FUI%2FRegister.pm%4013373&old=twiki%2Fbranches%2FMAIN%2Flib%2FTWiki%2FUI%2FRegister.pm%4013364

This code provided TWiki with an API to process a TWiki table line by line.

Is there something else that does the same?

Ultimately I want to create a set of topics, one per line, from a TWiki table. Just like Bulk Registration does.

-- MartinCleaver - 16 Oct 2008

I don't understand the question. You say there is code that processes TWiki tables line by line, and then ask if there is something that does the same. Why? Is there something wrong with the existing code?

-- CrawfordCurrie - 17 Oct 2008

Hi Crawford. I modularised the functionality into TWiki::Data::DelimitedFile so that it was separate and could be used by any part of TWiki. When I left it it was only used by registration.

You unmodularised it, folding the code into TWiki::UI::Register. So now the code is only accessible by registration. See also http://twiki.org/cgi-bin/view/Plugins/ExcelImportExportPluginDev#foo_47

Before I remodularise it, is there some other API in TWiki that accomplishes this goal?

Thanks, M.

-- MartinCleaver - 17 Oct 2008

Ah, OK. There's some history here. From an architectural perspective, we really need to get away from the plethora of island parsers/readers we have in TWiki. I originally looked at TWiki::Data::DelimitedFile with a view to re-using it wherever table parsers were required in TWiki. I also looked at using the CPAN equivalent, CPAN:Text::Delimited. However both these readers suffer from being very limited and plagued by "special cases" (for example, row extension using \, embedded | etc. etc.), and neither is event-driven, so I couldn't find any reasonable way to re-use either of them anywhere else. In the end I decided just to flatten out that one application of Data::Delimited (15 lines of code) because I wanted to discourage re-use of this approach elsewhere in the core.

My intention was to move towards use of an object model for topics (part of the TopicObjectModel) that would support a "table view", but this work has been stalled since the SF summit. What we really want to head towards is:

my $tom =  $session->{store}->readTopic(undef, $path, undef );
my $tit = $tom->tables(); # get tale iterator
while ($tit->hasNext()) {
    my $table = $tit->next();
    my $rit = $table->rows(); # get row iterator
    while ($rit->hasNext()) {
        my $row = $rit->next();

-- CrawfordCurrie - 18 Oct 2008

Summary TWiki::Data::DelimitedFile needs exposing in TWiki::Func
ReportedBy TWiki:Main.MartinCleaver

SVN Range 13364 - 13373
AppliesTo Engine
Component Registration
Priority Low
CurrentState Waiting for Feedback
WaitingFor TWiki:Main.MartinCleaver

TargetRelease major
ReleasedIn 4.0.2
Edit | Attach | Watch | Print version | History: r8 < r7 < r6 < r5 < r4 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r8 - 2008-10-18 - CrawfordCurrie
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback