Lyrics & Knowledge Personal Pages Record Shop Auction Links Radio & Media Kids Membership Help
The Mudcat Cafesj

Post to this Thread - Printer Friendly - Home
Page: [1] [2]


Digital Tradition Upgrade?

Related threads:
Dick Greenhaus birthday June 19 (10)
DT Attribution & Minor Corrections PermaThread (76)
Digitrad and Blicky (14)
History of the Digital Tradition (23)
Obit: Dick Greenhaus (1929-2019) (124)
Dick Greenhaus links on YouTube please? (5)
Dick Greenhaus and the Digital Trad (6)
How can I reach Dick Greenhaus / Camsco? (22) (closed)
What is the Digitrad? (6)
DT Alphabet (20)
DT Attribution & Minor Corrections - recorded msgs (361) (closed)
DT shared tune list (4)
Tech: Digital Tradition (3) (closed)
What is DT and where do I find it? (18)
Tech:Running Digital Tradition in Linux/Vista/Win7 (37)
Criteria for adding songs to database? (11)
Correct the Digital Tradition (57) (closed)
error - DT Keywords (17)
Chords/TAB in the Digital Tradition (5)
DT Tunes - Do they sound odd to you? (11)
How to Correct Songs in Mudcat Database (21)
TECH: Playback problems with New DT (11)
TECH: New Windows DT Midi problem (60)
Treasures in the DT (5)
Scholarship? new DT of any value? (38)
Downloadable Version of DT (9)
Help: DT Songs in ABC Format on the Web? (7)
Weakest Link: The Digital Tradition (72)
Macintosh Digital Tradition (13)
Indexing Digitrad songs (7)
Help: Who corrects the DT?? (22)
DT - version release? (11)
Help: boohoo - probs with dt database... (Mac) (4)
Help: Lyric adds to DT (5)
Digital Tradition Triumph (13)
Adding a song to the DT? (6)
Digital Tradition: How Does It Grow? (23) (closed)
Sale of DT intellectual property? (23)
DT search tips (5)
Netiquette for linking to DT lyrics? (4)
I LOVE the DT!!! (11)
How To Post MUSIC to DT (18)
DT Criteria? (3)
Printing (& gifs) of Digital Tradition songs? (4)
Tunes for DT songs? (13)
How can I get the tunes from DT? (5)


dick greenhaus 08 Jan 07 - 10:22 PM
GUEST,Jon 09 Jan 07 - 03:19 AM
GUEST 09 Jan 07 - 01:54 PM
Bill D 09 Jan 07 - 03:22 PM
Susan of DT 09 Jan 07 - 04:29 PM
GUEST,Jon 09 Jan 07 - 05:46 PM
dick greenhaus 10 Jan 07 - 01:10 PM
GUEST,Jon 10 Jan 07 - 01:44 PM
Ferrara 10 Jan 07 - 04:45 PM
Malcolm Douglas 10 Jan 07 - 10:03 PM
Ferrara 11 Jan 07 - 12:14 AM
The Fooles Troupe 11 Jan 07 - 12:21 AM
GUEST,Jon 11 Jan 07 - 04:54 AM
GUEST,Jon 11 Jan 07 - 05:08 AM
Tom Hamilton frae Saltcoats Scotland 11 Jan 07 - 10:13 AM
dick greenhaus 11 Jan 07 - 11:33 AM
GUEST,Jon 11 Jan 07 - 01:28 PM
dick greenhaus 12 Jan 07 - 10:33 AM
GUEST,Jon 12 Jan 07 - 10:42 AM
Mick Pearce (MCP) 14 Jan 07 - 07:41 PM
GUEST,Jon 14 Jan 07 - 08:08 PM
GUEST,Jon 14 Jan 07 - 08:12 PM
Linda Goodman Zebooker 14 Jan 07 - 11:40 PM
The Fooles Troupe 15 Jan 07 - 01:11 AM
The Fooles Troupe 23 Jan 07 - 07:14 AM
Andrez 19 Sep 07 - 09:02 AM
dick greenhaus 13 Oct 07 - 01:43 PM
Susan of DT 14 Oct 07 - 11:25 AM
dick greenhaus 15 Oct 07 - 10:24 AM
MMario 15 Oct 07 - 10:26 AM
John MacKenzie 15 Oct 07 - 10:56 AM
Dan Schatz 15 Oct 07 - 11:29 AM
Jeri 15 Oct 07 - 11:33 AM
Dan Schatz 15 Oct 07 - 11:35 AM
Jeri 15 Oct 07 - 11:40 AM
Joe Offer 17 Oct 07 - 12:50 AM
Susan of DT 17 Oct 07 - 08:53 AM
MMario 17 Oct 07 - 09:00 AM
Geoff the Duck 18 Oct 07 - 05:21 AM
MMario 18 Oct 07 - 08:49 AM
GUEST,Andrez 21 Oct 07 - 08:27 AM
Share Thread
more
Lyrics & Knowledge Search [Advanced]
DT  Forum Child
Sort (Forum) by:relevance date
DT Lyrics:













Subject: RE: Digital Tradition Upgrade?
From: dick greenhaus
Date: 08 Jan 07 - 10:22 PM

Believe it or not, content isn't the problem. And the thought of having different databases for online and stand-alone makes Susan and I shudder to the extent that I fear for the integrity of our home. The search capability of the online version is, IMO, pathetic.
    It's NOT simpler to just update the online version; it's much more difficult than it is to create a new stand-alone one, and then convert that to HTML. The problem of platform independence is one that we're close to licking. The tune handling is not.
The online DT is really a bastardized version of the real DT; created solely because, in the early days, Max had lots of storage space but had problems with processing time due to limited bandwidth. This, hopefully, will not be a limitation in the future.

If the object is simply to increase the number of lyrics available, it's not really necessary top do anything--the forum search can take care of that. I'd hate to see us degrade the product even more than it has been.


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon
Date: 09 Jan 07 - 03:19 AM

Dick, there are different ways of going about things and I'm not saying this is the right choice but in relation to what I've suggested, and in what we went through before, your comments aren't making any sense.

Believe it or not, content isn't the problem.

I'm not convinced it's not a problem but we will see when the new version is out.

And the thought of having different databases for online and stand-alone makes Susan and I shudder to the extent that I fear for the integrity of our home.

That is precicesly what you have now. The suggestion is to reduce this to one but to update the online and forget your stand alone one. This, should you wish, makes it easy to have a trused group of editors - a reduction in work load, not an increase.

The search capability of the online version is, IMO, pathetic.

That could be improved.

It's NOT simpler to just update the online version; it's much more difficult than it is to create a new stand-alone one, and then convert that to HTML.

Takes me about 30 seconds for me to generate a separate html page for each song, cross ref links to keywords, etc. It is not difficult from an SQL database.

The online DT is really a bastardized version of the real DT; created solely because, in the early days, Max had lots of storage space but had problems with processing time due to limited bandwidth. This, hopefully, will not be a limitation in the future.

Please explain.


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: GUEST
Date: 09 Jan 07 - 01:54 PM

There seems to be a bit of confusion regarding the various versions of the DT that are extant. The HTML version on the Mudcat site is not created independently--it's derived from a single master version which is also the basis for the Windows and Mac versions.
Yes, one generate a separate HTML page for each song--at 30 seconds per, that would take some 80 hours of steady work. The result would be pretty much what we have now--a database with a lot of useful information that's hard to find. It would still fail to solve the music playing problems that have been a constant nuisance.

When I say that content isn't a problem, it's just what I mean. We now have another 1000 lyrics ready to go, and finding more is no problem.

we're concentrating on the standalone version for a couple of important reasons: a)it's a much better product. b)once the standalone is ready, converting to a web-based database is trivial c) thae standalone is much easier to maintain--just try tracking down duplications of songs in the HTML version.


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: Bill D
Date: 09 Jan 07 - 03:22 PM

(I assume that's you, Dick..*smile)

question....when you say you have "another 1000 lyrics ready to go", does that include changes & corrections that folks have fretted about pretty regularly? Do you still mean that changes and updates would only be a once or twice a year happening? It would mean a lot to have the possibility of correcting errors regularly (read that as "almost immediately").


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: Susan of DT
Date: 09 Jan 07 - 04:29 PM

Yes, that was Dick at 1:54. I don't know why his cookie wasn't functional.

There has to be one master copy from which new versions, other versions, etc. are generated. Once you have some changes made to copy a and others made to copy b, you have chaos and no way to get them back into one place. The posted version cannot be changed on the fly. Changes go into the master from which the next version is produced.


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon
Date: 09 Jan 07 - 05:46 PM

No Dick, I am aware of the situation with the databases and I meant the run to produce all this takes about 30 seconds. I am aware that doesn't answer the search problem but I didn't suggest it did, I was just pointing out what you said was wrong.

Anyway, I will leave it at this, I've tried to point out some things you have missunderstood but I've no wish to fall out with you or come over as if I'm dictating which way you should go with it.


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: dick greenhaus
Date: 10 Jan 07 - 01:10 PM

Guest Jon-
Just out of curiousity, what did you use as a starting point for those HTML pages?


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon
Date: 10 Jan 07 - 01:44 PM

Dick, I'd already built a MySQL datbase from a text file imported from AskSam for this we talked a little about in the last thread on the dt.

I just ran a php script on the MySQL database to produce the HTML version. My coding isn't particularly good but here it is if you want to see how it was made from the database.

function microtime_float()
        {
        list($usec, $sec) = explode(" ", microtime());
        return (float) $usec + (float) $sec;
        }
        
function addkeyword($SongID)
    {
    global $file, $nl;
    $SQL ="Select keyword.KeywordID, keyword.Keword from keyword, songkeyword
    where keyword.KeywordID = songkeyword.KeywordID and songkeyword.SongID = " . $SongID;
    $result = mysql_query($SQL);
    $rs = mysql_fetch_array($result);
    $added = false;
    while (!($rs==0))
        {
        if ($added)
            fwrite($file, ", ");
        else
            $added = true;
        fwrite($file, "<a href='../keyword/" . $rs["Keword"] . ".html'>" . $rs["Keword"] . "</a>");
        $rs=mysql_fetch_array($result);
        }
    fwrite($file, $nl);   
    }


function addtune($SongID)
    {
    global $file, $nl;
    $SQL = "Select ABC, Title From tune, songtune where SongID = " . $SongID . " and songtune.tuneid=tune.tuneid";
    $result = mysql_query($SQL);
    $rs = mysql_fetch_array($result);
    while (!($rs==0))
        {
        fwrite($file, "<a href='../tune/" . $rs["Title"] . ".html'>" .$rs["Title"] . "</a>" . " ");
        $rs=mysql_fetch_array($result);
        }
    fwrite($file, $nl);
    }

function addkeywordsong($KeywordID)
    {
    global $file, $nl;
    $SQL = "Select Title, song.Filename From song, songkeyword
            where song.SongID=songkeyword.SongID and
            KeywordID = " . $KeywordID;
    $result = mysql_query($SQL);
    $rs = mysql_fetch_array($result);
    while (!($rs==0))
        {
        fwrite($file, "<a href='../song/" . $rs["Filename"] . ".html'>" .$rs["Title"] . "</a>" . $nl);
        $rs=mysql_fetch_array($result);
        }
    fwrite($file, $nl);
    }



require("config.php");
require("top.php");

$start = microtime_float();
$nl = "<br>" . chr(13).chr(10);

//add the songs
echo "adding songs<br>";
$SQL = "SELECT * FROM song ORDER BY Title";
$result = mysql_query($SQL);
$rs = mysql_fetch_array($result);
@mkdir("html");
@mkdir("html/song");
@mkdir("html/tune");
@mkdir("html/keyword");
while (!($rs==0))
        {
        $filename = "html/song/" . $rs["Filename"] . ".html";
        $file = fopen($filename, 'w');
        //echo $filename . "<br>";
        $SongID = $rs["SongID"];
        $Song = $rs["Song"];
        fwrite($file, "<b>" . $rs["Title"] . "</b>" . $nl);
        fwrite($file, $Song);
        fwrite($file, "<b>Keywords: </b>");
        addkeyword($SongID);
        fwrite($file, "<b>Tune: </b>");
        addtune($SongID);
        fclose($file);
        $rs=mysql_fetch_array($result);
        }

//add the tunes
echo "adding tunes <br>";
$SQL = "SELECT * FROM tune";
$result = mysql_query($SQL);
$rs = mysql_fetch_array($result);
while (!($rs==0))
        {
        $filename = "html/tune/" . $rs["Title"] . ".html";
        $file = fopen($filename, 'w');
        fwrite($file, str_replace(chr(13), $nl, $rs["ABC"]));
        fclose($file);
        $rs=mysql_fetch_array($result);
        }
        
//add the kewords
echo "adding keywords<br>";
$SQL = "SELECT * FROM keyword";
$result = mysql_query($SQL);
$rs = mysql_fetch_array($result);
while (!($rs==0))
        {
        $filename = "html/keyword/" . $rs["Keword"] . ".html";
        //echo $filename . "<br>";
        $file = fopen($filename, 'w');
        addkeywordsong($rs[KeywordID]);
        fclose($file);
        $rs=mysql_fetch_array($result);
        }

//make an index
echo "adding index<br>";
$indexfilename = "html/index.html";
$indexfile = fopen($indexfilename, 'w');

for ($I=65; $I<=90; $I++)
    {
    fwrite($indexfile, "<a href=".chr($I). ".html>" . chr($I) . "</a> ");
    if ($I == 65)
           $where = "Title <='B'";
    elseif ($I == 90)
       $where = "Title >='Z'";
    else
       $where = "Title >='" . chr($I) . "' AND Title <'" . chr($I+1) . "'";
    $SQL = "SELECT Title, Filename FROM song Where " . $where;
    $result = mysql_query($SQL);
    echo mysql_error();
    $rs = mysql_fetch_array($result);
    $file=fopen("html/" . chr($I) . ".html", 'w');
   while (!($rs==0))
        {
       fwrite($file, "<a href='song/" . $rs["Filename"] . ".html'>" .$rs["Title"] . "<a>" . $nl);
        $rs=mysql_fetch_array($result);
        }
        fclose($file);
    }
fclose($indexfile);
$end = microtime_float();$end = microtime_float();
echo "File built in " . round($end - $start,2) . "seconds";
?>


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: Ferrara
Date: 10 Jan 07 - 04:45 PM

Dick, I was suggesting making the online database the master file. The data in each new stand-alone release would be automatically generated as a "snapshot" of the on-line database at specified intervals, making new releases automatic once the basic code for the stand-alone database is in place.

I no longer have the skills to do any of this though.... However I know the Folklore Society can produce reports of its online data in machine-readable format that can be imported into various software.

Is this not possible for the online Digitrad database as well? Since the stand-alone version is being revamped, I would expect that whatever database engine is used would include an import feature? Lots of data of course but not a maintenance nightmare.

And, since the stand-alone database is released in specific versions, there would be no data concurrency problem. The stand-alone version would be a snapshot of the master database at a given time.

Last, about providing linked music files. I would be against having just lyrics and no tunes in the DT. That would be a huge step backward.

But I can't see any rationale for withholding a song from the DT because there is no tune available at present! What is the problem with having some songs without tune files until tunes can be made avaiable? Bill has a saying: "Fifty percent of something is better than 100 percent of nothing."

Rita F


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: Malcolm Douglas
Date: 10 Jan 07 - 10:03 PM

And, of course, many DT texts are copied from sources that didn't include tunes in the first place; some of these are 'Child' ballads for which there is no known melody in any case. My personal feeling is that it is better to provide no melody rather than provide one which may never have been used with the text in question, or which is not properly identified. DT files frequently link to unnamed, unidentified tunes that may be anything at all; no information is provided.

Were all those unidentified tunes really sent in without source information? Not the ones I provided (several hundred, I think), though the essential contextual info I included has frequently been omitted or misquoted, both in the DT and in the Mudcat midi annexe.

I agree with Ferrara. The source database needs to be an online version, converted to an easily updateable form in a current format (SQL rather than the obsolete AskSam). That isn't hard to do.

Once there is a reliable central resource into which corrections can be entered as they arise (instead of five or more years too late) standalone versions can be built from it at whatever intervals best suit. Trying to do it the other way round just isn't going to work any more.


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: Ferrara
Date: 11 Jan 07 - 12:14 AM

The model for having the online version be the master file is the basis of the concept of "distributed processing." The definitive version of an organization's DB is on a central mainframe but snapshots can be downloaded either at regular intervals or at need. Then they can be used locally in various ways.

Using this model for Digitrad, the snapshots would be downloaded into a new stand-alone database, by an automated process, at regular intervals. And distributed by whatever method you like.

The on-line version would be the only place where changes are applied, and there would be some kind of clear, managed process for applying and reviewing changes.

Rita F


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: The Fooles Troupe
Date: 11 Jan 07 - 12:21 AM

I regret not having stated clearer the same basic concept as Rita...


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon
Date: 11 Jan 07 - 04:54 AM

Malcolm, AskSam isn't obsolete. I belive it's good at what it does too but I would favour an SQL relational database over the proprietry flat file format.


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon
Date: 11 Jan 07 - 05:08 AM

Dick, out of curiosity, what problems do you have with my online search? It allows a full text search (natural) as well as matching words or a phrase, allows searches on dt keywords (I called them categories, and youenter them withouy the @) and a combination of both.


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: Tom Hamilton frae Saltcoats Scotland
Date: 11 Jan 07 - 10:13 AM

When I go to the digital keyword search and then click one of the boxes I get an error message, I mean it's been like that for months, since last year.

I even click the two boxes at the top of the page above the box that says search


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: dick greenhaus
Date: 11 Jan 07 - 11:33 AM

Guest John-
Which online search do you refer to? The one at Mudcat now a)doesn't recaognize phrases containing common articles, so that users keep reporting that they can't find songs. b)doesn't accept wildcards, so that variant spellings present problems c)can't deal withh Boolean relationships.

Malcolm-As long as DigiTrad accepts contributions from anyone, there's no way og guaranteeing "authenticity". Criticizing it for that lack is somewhat like criticizing the Roud index for not including lyrics. Or criticizing a camel for being a poor cow. In retrospect, I feel that we might have been better off not soliciting contributions and keeping a tighter control over what's in there, but if we had it would be a much more limited resource.

Rita-The delays in updating have nothing to do with which version is considered the master copy; it's been due to the several layers of volunteer programmers between us and the online version. What I'm working on now an arrangement that permits me to interact directly with the website, while enabling me to maintain a single platform-independent version. Which will cerainly include the tunes, in some form or other.


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon
Date: 11 Jan 07 - 01:28 PM

I was reffering to mine here Dick. It doesn't meet all your oobjections to the Mudcat one in one search method but get closer.

I'm curious about:

"What I'm working on now an arrangement that permits me to interact directly with the website,"

Please expand on this.


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: dick greenhaus
Date: 12 Jan 07 - 10:33 AM

Guest Jon-
I'm very impressed with what you've done. It's certainly a better implementation than the one we currently have on Mudcat.
Could we discuss this offline? My Email URL is dick@camscomusic.com; my phone number (US & Canada) is 800/548-FOLK (3655)


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon
Date: 12 Jan 07 - 10:42 AM

Certainly Dick, I've emailed you to make contact. (I won't be using the phone btw, I'm in the UK).


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: Mick Pearce (MCP)
Date: 14 Jan 07 - 07:41 PM

I've mentioned my approach to the design before, but I'll give a description of what I've been doing with the DT.

I still think the way forward is to use a relational database with the semantic structure of the song files. In my database I have tables for:

  Songs - with titles and identifier fields (Child, Laws, Roud, DT#)

  Keywords for Songs - this links the Song table to all the keywords for the song

  SongLines - the entire DT data, but with type added to each line eg Title, Song Text, Notes etc (I think I have 12 types)

  Tunes for Songs - this links the song table to all the tunes given for it

  Tunes - the individual tune names with links to the abc

  Abc - the abc lines for each tune.


Both the text data and the tune data were generated programmatically from the original DT files (in fact I think it's not the last release I've got loaded but the one before). There was a bit of manual processing aferwards (mostly when the heuristics for separating song notes from song lines in the DT failed. I added a switch button to my viewer so that I could go through the files and have selected lines switched from text to notes or vice versa. There were also a handful of multivoice SongWright files that I didn't convert to abc - the extra programming to handle it didn't seem worth it for the few files affected).


I've written a Java browser to access the database (I'm using Java for intended platform independence). The browser part displays all the titles in a tree (title name under start letter nodes) and I make use of the line type information when displaying selected songs - I keep the song notes separate from the song text. I can display the score (I'm currently using abcm2ps to generate the score and I've got a version of the Java postscript interpreter Toastscript that I've
modified to allow it to display and print the score from within the browser. I may automatically display the music in the browser later.

I have implemented a simple karaoke midi player in the browser that displays the song-syllables along with the tune, one line at a time. The midi is generated using abc2midi, so you can see I'm not totally Java yet! My personal opinion is that you don't necessarily need the karaoke to happen on the score the way the current DT browser does. While that's nice, it's a bugger to do unless you're implementing the score writer yourself (which I'm not, certainly not for now; even
later I'd prefer to generate postscript and forgo bouncing ball/highlighting the score), and I'm of the opinion that if you can read the music you don't need the karaoke and if you use the karaoke
you don't need the score.

For searching I allow searches against the following items:
  Identifier: Child, Laws, Roud, DT, exact titles, exact keywords
  Substrings of Titles or keywords
  Substrings of single lines of song text or song's notes
  Substrings in any line of a song text, any line of a song's notes, any line as in the original DT for the song

The search item can be a list of phrases separated by && (for and) or || (for or), each phrase optionally starting with ~ to negate the test (match-> no match, string in item -> string not in item). It's not particularly elegant, but I chose this because in Java it's almost trivial to split the individual items out. At present I can only apply the tests to a single item at a time - eg I couldn't seach for (Child=1) AND ('Nic Jones' In Song Notes). (Though I return the search result titles as a node on the Browse tree, so I could in principle apply a new search to songs only within a node - I don't at the moment, but it shouldn't be too hard to do. I do allow the results of several searches to be added to the result node or clear out the old entries first). At present I force all '&&' or all '||' to be used in a test, but that's because I couldn't be bothered handling parentheses to set the precedence of (A&&B)||C versus A&&(B||C). I could allow both now if I was willing to accept the default operator precedence, but choose not to. In theory there's no limit to how many operators can appear, but some of the whole song searches could take a long time with a lot of conditions. (All theses searches essentially only generate one of three types of queries - a simple one against the Song table, a simple join of Song with Keywords or Song with the DT Lines, a sequence of joins of Song with DTLines linked by INTERSECT or UNION). Also the conjunctions are not order dependent William && Mary would find 'William loves Mary' and 'Mary loves William' (though it would be trivial to allow something like 'William > Mary' to pick out 'William loves Mary' by creating a simple LIKE/regexp condition in the generated search; in fact I like that idea so much I might do it!. I'm not in favour of general regexp expressions being allowed though - it's too easy for people to get them wrong!).

So examples of searches I can do are:


Child#:       Child#: 4||8       finds all entries marked as Child #4 or Child#8
Title:         William && ~Mary   finds song titles containing William but not Mary
Song line:      Dilston && ancient find songs with both of these words in the same line
All song lines: Dilston && ancient finds songs with 'Dilston' in any line of the song and 'ancient' in any line of the song
(the last will find DERWENTWATER'S FAREWELL, the previous wont find anything)

Text searches are currently case insensitive, but the code generating the SQL query can already generate a case sensitive search if I ever add that as an option in the search strings.

This search set can run everything that my Windows DT version can do sslighly faster on some of the searches, but essentially the same order of magnitude), but can run the more complex searches too. Note that in the substring searches I search for any substring, not whole words. So 'Hall' will match 'shall' and 'halleluja' too. The database I'm using (Apache Derby - a pure Java database) doesn't support any whole word text searches . I could index all (non-trivial) words in the database and use those, but I don't find the absence of such searches a problem. There are relational databases that support document handling better than this - Oracle, SQL server and MySQL have support for word and phrase searches (I can't find phrases that span two lines for example) and I may try to do this with my MySQL version of the database (the Java code can access this just by selecting a different database driver, so I shouldn't need to change anything else to run my current version - I currently have the data in Apache Derby, MySQL and MS Access, and could run the browser from any of them - I could also have it as a client to a server of my Derby DB - though I'm usually running on Derby, part of my heading towards an all Java system. I wanted to develop a system that used only free components and for relational databases both MySQL and Derby are free and I can use either from Java, Derby just has a smaller footprint - about 2Mb for the code.)

I haven't added the editor form yet, but that's essentially just putting a lot of fields on a form (I've designed it, just not programmed it) and updating the database from it. My ultimate aim is to use it to add my own songs and tunes to it (I'll just flag DT orignal files separately). I'll also extend the tune info for a song to include links to external Audio files (midi/wav/mp3) and let me be
selected for the song along with the current tunes available.

This is more technical stuff than I'd normally post, but I thought you might be interested.


For the DT I think there are more significant things that can be done to simplify the administration of the system and increase the usefulness of the online database.

Having a proper Add Song form on Mudcat would be my choice for the most important change to the system (moreso than the browser system). At the simplest I'd include Titles, Song Text and Notes plus the Identifier information. It could be stuffed as is into the online database - it needs no more than a flag field to distinguish it from the last released DT.

The searches available could include the songs from the start and the editors could easily locate the new songs for tidying up and possible aapproval for inclusion as release songs (or even deletion - keep an archive note - "Song entered but duplicate of...". It would then be trivial to issue updates to the distributed versions of the DT. The song could remain 'open' for annotations - where people could post possible corrections to the song. These could be displayed along with the notes when displaying the song and again the editors could review these when preparing a new release (or just ongoing - you'll always know when there are open annotations) and either alter the song or leave them as notes. I'd do the same for tunes too.


But as I've said before whatever way the DT goes is fine with me. It's a great resource and it's been provided free by people who put in the time and effort just for the love of it. While I've written my own browser (partly just to get the hang of Java, partly to get some things the way I'd prefer them, and still work in progress, though I have found Java very quick and easy for development - although the browser's not finished, I've probably not spent more than a week on creating it), the (Windows) version that came with the DT is fine (well it's annoying having a new window open for each song you look at - that's one change I did make: the song select/search result tree is on the same page as the song display, I only open a new window for the score view/print). And despite the disproportionate space writing about my browser and its searches compared to my ideas for the online system, it's that last one that I'd really like to have considered.

Mick


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon
Date: 14 Jan 07 - 08:08 PM

Mick, you would probalby take things a lot further than I would. Still think a Mudcat led soulution is the ideal though.... ~(Or at least a no I can't/won't from Max would be helpful).


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon
Date: 14 Jan 07 - 08:12 PM

(more helpful than a "no man's land that is". I believe they could do it...


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: Linda Goodman Zebooker
Date: 14 Jan 07 - 11:40 PM

I appreciate reading these technical postings. I can't begin to understand them, but I can get a small sense of what's underneath something like the digitrad - it's less of an opaque mystery to me now.
Linda


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: The Fooles Troupe
Date: 15 Jan 07 - 01:11 AM

MCP

You seem to have done a lot of useful stuff - it also would port to Linux easily, as far as I can tell.

With a separate DB each user could add and sort their own compositions.

Robin


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: The Fooles Troupe
Date: 23 Jan 07 - 07:14 AM

FYI: (those interested only)

The ABC Plus Project         

here at sourceforge

has source and binaries for various ABC related things WIN exes & Linux rpms for ABC Plus - which was an intended upgrade to allow harmony as well as the original intended melody that ABC was designed for.

Making Music with ABC Plus
Some ABC Plus choral music
An ABC package for LaTeX
abcm2ps binaries
abcm2ps extensions
abcMIDI binaries
tclabc binaries
NoteEdit
ABCEdit
JedABC
abcpp
abc2prt

Of interest is NoteEdit: a free, great KDE (Linux) program for editing music visually. It imports MIDI and exports in MusiXTeX, Lilypond, PMX, MIDI, and now ABC Plus too! Its original author, Joerg Anders, wrote the ABC Plus support in an amazingly short time after my suggestion/request. Kudos to Joerg!

Recently, Joerg stopped developing NoteEdit, but other developers have taken over: http://noteedit.berlios.de/. Thanks a lot, folks!

also

ABCEdit is a Windows program that integrates abcm2ps, abc2midi and GhostView in a single package. Very easy to use, and it's freeware! Kudos to Joop Coolegem, .

ABCEdit home page is http://www.abcedit.tk/.

Also

JedABC is an extension to the Jed editor that turns it into a powerful and easy to use IDE (Integrated Development Environment) for ABC files, with full integration with preprocessors, converters, players and previewers. It's similar in concept to BarFly or runabc, with several advantages.

JedABC helps you write ABC files with useful features:

All of these are close to what has been claimed is wanted.

There seems to be no 'words' or 'bouncing balls' as yet, but if approached by the Digitrad Development Team, I am sure that the editors/maintainers of these programs would give thought to those extensions.


Robin


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: Andrez
Date: 19 Sep 07 - 09:02 AM

Are we there yet?

Is there any chance of a cross platform beta version of the DT with cool text search capabilities any closer to existence?

Would it be possible to have an update on progress, such as it may be since this thread was last activated last January? If there has been some progress I'd be interested to know what if any of the suggestions made in this thread or the others referenced by guest Jon: Tech: Digital Tradition Programmer Needed, been taken on board and developed? Has there been any other behind the scenes progress?

Is there anything anyone can do to help progress the issue: skills, time, money? If skills or time is still an issue could we take up a collection? What would it cost to buy the skills and time? I'd be happy to chuck in a hundred bucks to start off a Digitrad development fund.

Cheers,

Andre

PS: Would it be possible to have a permanent thread where progress notes on the new Digitrad can be posted every now and again?


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: dick greenhaus
Date: 13 Oct 07 - 01:43 PM

We're very close to having a platform-independent version of the DT with vastly improved search functions. It includes the capability of frequent single-file updating, addition and deletion, which means that "model years" will be a thing of the past.

We haven't solved the tune problem, however. What's needed is a program that will accept SongWright or ABC files (with lyrics), and play them (preferably with a synchronized "bouncing ball" or some such place marker). Preferably (though not an absolute necessity) it should permit display and printing of the score ("dots".) This program could be integral with an on-line DT, or could be a downloadable program to be run at the user's computer, with the DT providing the input tune data only (this would reduce bandwidth requirements at Mudcat's server considerably.)

I know that some of you have worked on this aspect, with varying degrees of success. I'd greatly appreciate hearing about the specifics of what you've done, and how we can adapt it. Remember, please, that the bulk of the users of DigiTrad are not programmers; it's important to make any such program transparent to the end user, who'd really like to click on a link on the lyrics page to hear and see the music.


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: Susan of DT
Date: 14 Oct 07 - 11:25 AM

refresh


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: dick greenhaus
Date: 15 Oct 07 - 10:24 AM

No interest at all? Hmmm.


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: MMario
Date: 15 Oct 07 - 10:26 AM

Dick - I think this will need to be kept "up" for a while to allow those with the specialized knowledge see it. If I recall - most of those are not among the heavy users.


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: John MacKenzie
Date: 15 Oct 07 - 10:56 AM

I'm trying to lose weight Leo!


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: Dan Schatz
Date: 15 Oct 07 - 11:29 AM

Do you think it would help to repost the request with a "Tech" prefix? Maybe it would catch the attention of the more computer savvy amongst us.

Dan Schatz


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: Jeri
Date: 15 Oct 07 - 11:33 AM

Dan, the more savvy posters posted in the previous 'Tech' thread to which Jon linked.


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: Dan Schatz
Date: 15 Oct 07 - 11:35 AM

Oops. That'll teach me not to read only the bottom bit of a thread.

Dan


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: Jeri
Date: 15 Oct 07 - 11:40 AM

It's not just you, Dan. The previous offerings seem to have fallen through the cracks. Personally, I don't know much about music programs, but some in that other thread sure did.


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: Joe Offer
Date: 17 Oct 07 - 12:50 AM

You know, I'll betcha somebody would like to know that Jeff put the 2007 online edition of the Digital Tradition is now available here at Mudcat. This includes additions that have been made since 2002, but I don't know all the edition dates. I do know that you can search for Feb07 and find the stuff that came out this year.
There's a lot of manual labor involved in this, to keep our old links to Digital Tradition songs working.
Thank you very much, Jeff - and thanks to Dick and Susan for the work they do constantly, putting together all the information. They've added hundreds of songs.
How many songs in the database now, Dick?
-Joe Offer-


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: Susan of DT
Date: 17 Oct 07 - 08:53 AM

Thank you, Joe. We did not even know it was up there. I wonder how long it has been there? Isn't communication wonderful?

Thank you, Jeff!

There are 10,000 songs in the Feb07 edition, vs. 9,000 in the 2002 edition, roughly. We are at work on the next edition. I just cleaned up (for the next edition) the obvious problems from running this list for "Feb07" - capitalization in titles, duplicate files, lack of version number in titles, incomplete files.
    I posted my notice less than an hour after Jeff posted the DT - but since I posted to an existing thread, I guess nobody noticed....
    Good thing MMario started a new thread.
    -Joe Offer-


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: MMario
Date: 17 Oct 07 - 09:00 AM

hey! Doesn't this deserve a new thread? Yes! It does!


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: Geoff the Duck
Date: 18 Oct 07 - 05:21 AM

The old Tech tread about Digitrad and Programmers has just resurfaced with a comment worth note . Here is a direct link to the posting -
BLICKY.
Quack!
GtD.


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: MMario
Date: 18 Oct 07 - 08:49 AM

I just thought the notice desserved a thread of it's own Joe; not trying to trump anyone


Post - Top - Home - Printer Friendly - Translate

Subject: RE: Digital Tradition Upgrade?
From: GUEST,Andrez
Date: 21 Oct 07 - 08:27 AM

Great to hear news of progress Dick. It would be easier to keep track of progress and catters contributions if there were a single perma-thread that consolidated the other threads. After no response to my query in Sept I thought the whole issue had been swept into the too hard basket until I saw the Digitrad & Programmers thread briefly revived a few nights ago. Please, please make this thread more accessible and not so missable!

Cheers,

Andrez


Post - Top - Home - Printer Friendly - Translate
  Share Thread:
More...

Reply to Thread
Subject:  Help
From:
Preview   Automatic Linebreaks   Make a link ("blue clicky")


Mudcat time: 20 May 9:19 AM EDT

[ Home ]

All original material is copyright © 2022 by the Mudcat Café Music Foundation. All photos, music, images, etc. are copyright © by their rightful owners. Every effort is taken to attribute appropriate copyright to images, content, music, etc. We are not a copyright resource.