To Thread - Forum Home

The Mudcat Café TM
https://mudcat.org/thread.cfm?threadid=97655
91 messages

Digital Tradition Upgrade?

31 Dec 06 - 09:38 PM (#1923687)
Subject: Digital Tradition Upgrade?
From: GUEST

Any possibility of this in 2007?

Just thought I'd ask, what with it being a new year and all.


31 Dec 06 - 10:07 PM (#1923694)
Subject: RE: Digital Tradition Upgrade?
From: wysiwyg

Yes, thanks-- as soon as you can get to it, Guest, that will be a great help!

~S~


01 Jan 07 - 06:27 PM (#1924315)
Subject: RE: Digital Tradition Upgrade?
From: MMario

Dick?


01 Jan 07 - 06:59 PM (#1924344)
Subject: RE: Digital Tradition Upgrade?
From: Greg F.

Send in those checks, money orders& cash & then ask again.


02 Jan 07 - 08:58 AM (#1924694)
Subject: RE: Digital Tradition Upgrade?
From: Susan of DT

Money is not the issue, Greg. Contributions go to Max/Mudcat for equipment and expenses in running mudcat. The Digital Tradition is separate and does not really have expenses.

We hope there can be a new version of the DT this year, probably not in the first six months of the year. Dick will write a longer response with more specifics on the issues (working on multiple operating systems, tune format that links words to notes without requiring that current tunes be reentered, fast enough searches and tune playing to work online for the volume mudcat gets, etc.) The volunteer who had done some of this programming has been working on another folk project for the last several years and has not had time to work on this.


02 Jan 07 - 09:34 AM (#1924728)
Subject: RE: Digital Tradition Upgrade?
From: dick greenhaus

There are a few technical obstacles that have prevented the publication of an updated DigiTrad. It's not a question of content--we have a thousand or so new songs waiting for publication. Problems are:

a) Operating systems. Whatever we release must be able to run on all versions of Windows (from 3.1 on up to, probably, Vista) and on all Mac operating systems that Mudcatters are apt to have. For several editions, the Mac versions have been supplied by a hard-working volunteer who is very busy working hard on other projects; I am a MacIlliterate, so this presents real problems. It's also necessary for any new edition to run using HTML, which is doable, but takes more time than I've had to spend.

b) Tunes. Our concept has always been to make the melody lines playable, with some device for showing where the words fit the tune (either bouncing balls or synchronized display or color highlighting or sune such scheme. More recently, the gentleman who worked out the Mac version did some neat programing that displayed the score (dots) and permitted playback in different keys, tempi and instrumentation.
It wasn't a perfect system (timing of notes was sometimes a bit wonky) but IMO a quite good one. Unfortunately, this program took as input transcriptions I made in a handy, if currently out-of-production, system called SongWright. Songwright is still the handiest entry program I've found, but it doesn't support playback (it did, playing the tune on a computer's beeper, but computers for the past six or seven years haven't come with beepers).
    SO, what's needed is a playback program that will work on PCs and Macs, and will play the tune, synchronize words and tune, permit one to print the score (and, ideally, permit changing of key, tempo and instrumentation.) ABCs seem to work--sort of. Does anyone know of an implementation of ABC that will do what's needed? MIDI doesn't handle words. Noteworthy may be a possibility--I'm investigating it. Whatever we come up with, I then have the tasks of converting some 4000 tunes to a new format--I don't want to have to do that more than once.
    If anyone likes to do this kind of programming, I'll be happy to supply both the specifications of a Songwright file (it's similar, though not identical to ABC) and grateful encouragement. At the moment I'm somewhat buried in trying to set up a new website for CAMSCO Music, and while I'm close to finishing that, it still takes up most of my spare time.


03 Jan 07 - 06:18 AM (#1925477)
Subject: RE: Digital Tradition Upgrade?
From: Hrothgar

MacIlliterate, Dick?

Does this mean that your father couldn't read and write?


04 Jan 07 - 03:00 AM (#1926249)
Subject: RE: Digital Tradition Upgrade?
From: The Fooles Troupe

Dick,

Linux is getting ever more popular, and what with VISTA having its greedy little DRM fingers over everything HW & SW wise (including screens!), some people such as myself see no need to waste all that money on VISTA in future - Linux will run on much smaller, slower, and cheaper HW.

If no one else is interested in working on a suitable Linux version, perhaps you and I can have some private words offline. 'Linux Version' may be a bit more hassle than it seems on the surface unless it will be able to be built to run on any Linux setup, of course...

All I would wish to say here is that it would be easiest, if that version could also use the same database and formats as the Windows versions, of course... :-) some executable will have to be built/found...

Robin


04 Jan 07 - 05:42 AM (#1926299)
Subject: RE: Digital Tradition Upgrade?
From: Mick Pearce (MCP)

Dick

To say that midi doesn't handle words isn't quite accurate. There are several types of text events that can be embedded in a midi file. In fact abc2midi will embed the lyrics from an abc file into the generated midi file and some players (Jet for instance will do this) will display these as the tune plays. I even wrote a trivial version in Java to do this - it just had to listen for a midi text event as the file played and display the word (or highlight it in a line of text with a little bit more work).


Mick


04 Jan 07 - 04:28 PM (#1926720)
Subject: RE: Digital Tradition Upgrade?
From: dick greenhaus

Mick-
Details, please? I know about abc2midi, but I'm not familiar with Jet. In fact, I can prolly write something that will convert Songwright to abc--they're pretty similar in concept abd structure.


04 Jan 07 - 05:01 PM (#1926745)
Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon

I can't think of anything to say that wasn't said last year in Tech: Digital Tradition Programmer Needed.


04 Jan 07 - 05:11 PM (#1926751)
Subject: RE: Digital Tradition Upgrade?
From: Bill D

I recently discovered a (free) midi player that WILL display words, moving a highlight as it goes, as the song is played, much like Noteworthy will highlight the notes. (It shows any text associated with the file)

VanBasco's Karaoke Midi Player

If this helps anyone, or would be useful in thinking about all this, it would be nice.


04 Jan 07 - 05:15 PM (#1926762)
Subject: RE: Digital Tradition Upgrade?
From: GUEST

Harmony Assistant and Melody Assistant are good with lyrics and abcs/midis


04 Jan 07 - 07:23 PM (#1926849)
Subject: RE: Digital Tradition Upgrade?
From: Mick Pearce (MCP)

Dick

Just to be clear - these are players that will show karaoke type word display synchronised with the music as the midi file plays, there is no score display like the current DT player. JetAudio is just a free audio player that I use as my standard audio (and video) player (I removed Windows Media Player from my system), but there are a host of players that will display the words from a midi file with embedded lyrics - VanBasco's mentioned above, KarPlayer, a java one called Jokey are some that I've looked at in the past (though some require a slightly different text embedding from the one produced by abc2midi). (I think Windows Media Player does it too, but it's so long since I had it on my system that I've forgotten).

Mick


04 Jan 07 - 08:37 PM (#1926890)
Subject: RE: Digital Tradition Upgrade?
From: dick greenhaus

There are lots of programs that will do any two of 1)play 2)display words synchronized with playing and 3) display score. I'd really shoot for one that does all three. It's possible; I just don't know how to do it.


05 Jan 07 - 01:20 AM (#1927048)
Subject: RE: Digital Tradition Upgrade?
From: GUEST

Harmony Assistant does all three - it even has a voice synthesiser which will 'sing' the lyrics to you! Written b Didier and Olivier Guillion and published by Myriad.
Here: http://www.myriad-online.com/en/index.htm


05 Jan 07 - 06:19 AM (#1927178)
Subject: RE: Digital Tradition Upgrade?
From: The Fooles Troupe

Well I think that looks pretty good, and even pretty cheap for a bought program - but now what for a Linux equivalent?


05 Jan 07 - 06:31 AM (#1927183)
Subject: RE: Digital Tradition Upgrade?
From: GUEST,Captain Ginger

Trouble is, can Monsieurs Guillion be persuaded to make the working parts of Harmony Assistant public domain to be bundled with the DT?
The alternative would be the possibly heretical concept of making the DT itself shareware (with any monies going to pay the licence for any play/display software, and to fund future updates). It's such a fantastic resource that it has to be worth ten dollars of anyone's money - and, of course, the online version is still freely searchable.


05 Jan 07 - 06:41 AM (#1927187)
Subject: RE: Digital Tradition Upgrade?
From: The Fooles Troupe

So what we need then is a GNU licenced program...

which can be compiled to run under Windoze or Linux, or for that matter any OS whatever...


05 Jan 07 - 07:52 AM (#1927221)
Subject: RE: Digital Tradition Upgrade?
From: Andrez

Re Dicks reference to technical obstacles above I would just like to pick up on the following points and contribute a few suggestions to the discussion.



The question here is why? Why not try and simplify things so that the upgrade works with relatively modern operating systems instead of trying to cater for everything.

From the Microsoft side that means forget about all Windows OS support prior to say at the very least NT/ Win 2000 through to XP. Vista schmista, it'll be years before (a) they sort out the bugs with that OS and XP will be compatible (well more or less we hope) and (b) before people learn how to really work within that environment. Leave Vista for a later upgrade again.

From the Mac side forget about offering support for Classic (Mac OS 9) and just focus onMac OSX. and its nice Unix underpinnings. Opensource Linux also gets the thumbs up but I'll leave it to someone else to argue which distro would be the best development environment for that OS.

The issue here is that the major software manufacturers dont support their old operating systems so why should DT developers. Mudcatters or anyone else who is still using Win 3.1 need to be encouraged to spend a few bucks and upgrade. I mean you can buy a P3 running W2k or XP for around $AUD100. A P4 isnt much dearer and either is superior to 3.1 0r Win 98 etc.

Regarding the next point:



The concept is great of course but once again looking for simplicity, instead of taking the programming route why not take the visual animation route and look at a Flash based approach where you can place images, text audio on a timeline and then animate them. You can also add a whole slew of pefab effects or develop custom ones to enjoy all of the visual bells and whistles.

The resultant file can them be playable on any hardware and operating system capable of running the specific version of Flash player. This way you bypass much of the separate platform specific programming and all of the hack work and time that goes with it.

Finally before anaay work in Flash, it would be even simpler just to put together a quick and dirty version of the DT as a stand alone web app. My 13 year old daughter could do that! You may not get all of the frills but getting a set of words along with an image of the notation and a link to a basic audio file would be a bloody good start to getting a new version of the DT out there.

Reminds me of the old zen line that Donovan adapted... First there is a mountain, then there is no mountain......... its just how we look at things.


05 Jan 07 - 07:57 AM (#1927225)
Subject: RE: Digital Tradition Upgrade?
From: Andrez

Hmmm, for some reason the two sections from Dicks post that I inserted didnt get included with the last post. Rather than take up space by reposting the whole thing, I'll just add the key lines that were omitted from my post below.

(a) Operating systems. Whatever we release must be able to run on all versions of Windows (from 3.1 on up to, probably, Vista) and on all Mac operating systems that Mudcatters are apt to have.

(b) Tunes. Our concept has always been to make the melody lines playable, with some device for showing where the words fit the tune (either bouncing balls or synchronized display or color highlighting or sume such scheme.

[left bracket caused those lines not to appear]--clone


05 Jan 07 - 07:59 AM (#1927227)
Subject: RE: Digital Tradition Upgrade?
From: Andrez

Still not there...........aaaaaaaaaaaaarrrrrrrrrrrrggggggggghhhhhhhhhhhhh!

[fixed---clone]


05 Jan 07 - 08:38 AM (#1927254)
Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon

Andrez, if there is to be a player/browser dt app, Java wins hands down for me. It seems far too much to have tailor made solutions for each O/S.

FT, I think you would tun into problems with the graphical side and perhaps sound side if you went down the route of something like a C program to compile for different plaforms.


05 Jan 07 - 08:47 AM (#1927264)
Subject: RE: Digital Tradition Upgrade?
From: GUEST,Captain Ginger

For a true cross-platform app, Java has to be the simplest. Although HTML would also be pretty universal (which probably shows just how little I know about compiling software!).


05 Jan 07 - 09:02 AM (#1927278)
Subject: RE: Digital Tradition Upgrade?
From: MMario

well guys - Dick is trying to keep things down to minimum hassle from his end; and (I suspect) also trying to keep his dependence on the "kindness of strangers" to a minimum as well.

Don't forget that even if conversion can be automated - at least the first conversion should be pretty thouroughly proofed and double-checked. That alone is going to be a massive undertaking.


05 Jan 07 - 09:17 AM (#1927291)
Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon

I think Mick (MCP) already has a conversion program.

While not tested propery or completed, I can get most dt SongWrite into abc here.

Yet Another Digital Tradtion can get it into abc and PMW.

I don't think that's a problem.

The problems the way I see it are defining whether there is a need for a player/browser app and which platform. Agreeing on a tune format to use, perhaps a live Mudcat version etc.

Apart from finding a developer if needed for poss one app, the main barrier IMO is the "Mudcat/DT loop".


05 Jan 07 - 12:19 PM (#1927458)
Subject: RE: Digital Tradition Upgrade?
From: OldPossum

As far as Harmony Assistant or Melody Assistant from Myriad is concerned, they have a freeware player that will play back music files in a browser. All you need to do is download a player from their website. Said player could also be bundled with Digital Tradition, with permission from Guillon brothers of course. They seem to be quite approachable.

The real problem is finding someone with Melody Assistant or Harmony Assistant who is willing to convert 4000 tune files! Personally, I have Melody Assistant, but I am not making any promises ...


05 Jan 07 - 12:26 PM (#1927465)
Subject: RE: Digital Tradition Upgrade?
From: MMario

It the "Assistants" can take abc input as claimed - thiat might be a solution - as Dick has said, conversion to ABC from songwrite looks plausible.


05 Jan 07 - 12:34 PM (#1927470)
Subject: RE: Digital Tradition Upgrade?
From: OldPossum

Yes - but I think that their player application can only play the native Myriad format. ABCs and MIDIs have to be imported to the native format first - and I am not sure that batch conversion is possible. But then again, there would be no harm in asking the Guillon brothers if they would like to somehow co-operate with Digital Tradition on such a project. They are a small independent company, and the project might be worthwhile for them, as a way of advertising their products perhaps.


06 Jan 07 - 12:40 AM (#1927968)
Subject: RE: Digital Tradition Upgrade?
From: Malcolm Douglas

Dick:

Most people here are talking about the stand-alone, downloadable version of the DT. That is, I think, of minuscule importance as compared to getting the online version into a condition where individual entries can be updated on a day-to-day basis as opposed to en bloc (and with few errors corrected but plenty of new ones added) every two or three years.

Mick and Jon have already made recommendations on that score, and I am convinced that that is the way to go. Many people have offered practical help over the years, but this has rarely been taken up. There are corrections to fundamental mistakes in the DT that have been sitting in the Forum for five years and more but have still not been incorporated into the database.

I do understand that it's difficult to abandon a setup that works (or used to work) and surrender a degree of control, but that is the only way to ensure that the DT does not become obsolete or, worse, that it becomes the single biggest source of misinformation about traditional music available in the entire world. It is rapidly heading that way.

The fact that you never intended the DT to be anything more than an online songbook is beside the point. It has grown beyond your intent, and whether you like it or not it is assumed to be, and is used as, an authoritative resource.

It is vital that corrections posted in the Forum (which, although it also contains enormous quantities of misinformation, is still one of the best resources on the web for accurate background on folksong; the problem being that you have to know quite a bit already to be able to tell who is talking sense and who rubbish) be added to the DT.

This can't realistically be done by two people. You need a viable database format which can be edited -and updated- piecemeal via an online interface. You need a team of people you trust to do that. You need to forget about adding new material until the material you already have is accurate, properly proofread and complete (with proper sources and attributions).

Until you have all that, there is no point at all in worrying about how to put together a cd version with a novelty "bouncing ball" feature.


06 Jan 07 - 06:14 AM (#1928069)
Subject: RE: Digital Tradition Upgrade?
From: Andrez

As with any project you need a plan. From a technical perspective, the DT project is really no different to any other standard IT type project. You need to identify what you want to achieve, look at the tasks involved, then link these tasks to a timeline and then find the people and resources to implement the project..

Some things can happen concurrently. For example the checking, editing, proofing or whatever... of the actual DT data. This is an essential task and interested people with the time, skills, resources etc, could be working on that element of the DT project.

On the other hand there is nothing in principle to stop other people from working on other elements like the database format and online interface. You just create a "shell" and use dummy data making sure all the bits of the database are connected and work as expected. When the data is ready, you populate the database, test and then deploy it for review and comment. Alpha and Beta testing in that way is all about quality and you get a better final product.

A cross platform solution in Java or Flash, Shockwave etc, could also be investigated and developed concurrently by interested 'catters. I'm sure a little bit of server space could be found as a test site.

Needdless to say you need people to make it happen. Why not put it out to interested 'catters who have the skills interest, resources, time and love of the music to form a working project team to identify the key tasks and sub tasks, develop a work plan, timeline and allocate people to specific subtasks and then get on with it.

This neednt be bigger than Ben Hur but could be managed by a small, committed team that is focussed on outcomes i.e. a new, updated DT available to all on line or in some stand alone format.

Cheers,

Andrez


06 Jan 07 - 06:40 AM (#1928080)
Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon

Andrez, before you start, you need to understand the situation. Please read the other thread I linked to in detail.


06 Jan 07 - 07:41 AM (#1928109)
Subject: RE: Digital Tradition Upgrade?
From: Andrez

OK thanks Jon, I checked that thread and recall that I had followed it previously as well. Given the fact of the quality of that discussion re the technical issues there is nothing I want to add in this thread.   

I do note that there didnt really seem to be any resolution from the earlier thread in terms of decisions about the direction to proceed from the technical perspective that is... unless there is something else I have missed (quite likely).

Also, given the obvious expertise of some of the contributors in that thread, it makes no sense to me as to why their skills havent been tapped and why some version of the new DT isnt in process. Have there been any decisions about the preferred direction since the previous thread? Dicks comments above would suggest not.

If a DT project does get underway and I can contribute my own IT skills in some small way I'd be happy to do so.

Cheers,

Andrez


06 Jan 07 - 08:35 AM (#1928141)
Subject: RE: Digital Tradition Upgrade?
From: The Fooles Troupe

Well, perhaps a 'HTML Version' might be the quickest way to getting a minimum (no bouncing balls!) setup across all platforms.

Flash players (depending on the version of Flash used!) are available for most OS platforms, I think.


06 Jan 07 - 08:52 AM (#1928152)
Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon

Again, we can do that. I've pretty well got that in a crude form here. Like many of the things talked about it is another simple matter to do IF...

What none of us can do is see if anything can be sorted between Mudcat and the dt.


06 Jan 07 - 08:54 AM (#1928153)
Subject: RE: Digital Tradition Upgrade?
From: The Fooles Troupe

Oh, unless a 'minimalistic form' of HTML is used, things will bloat - 'MS Word HTML' ---- aaaargh!

Oh, and while I suggest Flash as a possibility, frnakly it's also a 'large format' too...


06 Jan 07 - 04:00 PM (#1928542)
Subject: RE: Digital Tradition Upgrade?
From: Bill D

I think Malcolm Douglas has a point. Although it may represent a lot of trouble, NOT heading that way may be more trouble in the long run. Pinning updates on one volunteer who may or may not be available to do a 3-4 version conversion seems shaky.

Something cross-platform, with easy updating by a chosen crew of trusted helpers looks like the way to plan, in my (admittedly amateur) view.


06 Jan 07 - 05:16 PM (#1928608)
Subject: RE: Digital Tradition Upgrade?
From: dick greenhaus

Okay, folks.
A couple of points:

A cross-platform approach is obviously the way to go. Unfortunately, all the cross-platform approaches I've seen would seriously degrade the usefulness of the collection.The online DigiTrad has a much weaker search engine than do any of the stand-alone versions; the Windows and Mac versions are much weaker (albeit prettier) than the DOS version.
    Susan and I would be absolutely delighted if some person or persons could take the programming aspect of DigiTrad off our hands. THat's what happened several years ago when the Windows and Mac versions were developed by a very helpful third party. Problem was, development of these versions involved a good deal of programming expertise and time, and when the pressure of outside work became sufficiently severe, the time required for an update to suit the new "improved" Mac and Windows operating systems wasn't available.
    It's easy enough to set up a program that will let one find lyrics by title--it's also a fairly useless exercise. Titles are an awful way to search for things. Full text and keyword searches are, in Susan's and my opinion, a necessity. So is the requirement of providing synchronized words and music--many of our users don't read music (either dots or abc), and realizing how the tune fits the words is by no means always obvious. These aren't frills--they're an integral part of the DT's concept. I'd love to be able to correct attrivutions and notes (and even words, though I'm not at all sure that a folk song has any "correct" words); doing this on a Wikipedia-type approach doesn't seem to me to be the way to go. Many, if not most, of the attributions and corrections we receive are inaccurate or trivial; editing is required.
    This being said, we'd be most grateful if some knowledgable folks could take over the programming part of DigiTrad; it's the part of the work that neither of us enjoys terribly, nor it is one in which we can claim any particular expertise. Nothing would make us happier than being able to go back to providing semi-annual updated editions, and concentrating on the songs. I'm reachable here, or by PM or at dick@camscomusic.com
    Don't all volunteer at once.


06 Jan 07 - 05:57 PM (#1928636)
Subject: RE: Digital Tradition Upgrade?
From: Bill D

Dick...would it be possible or useful if the lyrics database were totally separate from the tunes? That is, make a list of midis (and alternatives, when known) that was not physically linked, but was easily available to those who needed a tune? Synchronized words & music are nice, and they could be done as midis using NoteWorthy, then made available for use in something like VanBasco's. No, it's not the total solution, but as you say, the Windows & Mac versions are not as searchable as DOS right now. A midi, even if it's NOT synchronized, is at least playable on a Mac.

(just brainstorming here...the important goal for many is to make updating more than an annual or semi-annual event....and not totally dependent on you & Susan having time & energy.)

You say you don't trust a wiki database...but these 'can' be done with limited access, so everyone whose grandma sang it "this way" couldn't make radical changes. The Mudcat forum itself can be edited by a few volunteers with limited access....

(The alt.comp.freeware online Wiki requires a password to access, and though it is easily available, that needn't be the case.)


06 Jan 07 - 06:04 PM (#1928642)
Subject: RE: Digital Tradition Upgrade?
From: Joe Offer

Dick, in the meantime, I wonder if it would be worthwhile to go back to issuing a DOS version of the Digital Tradition on a regular basis, perhaps along with an HTML copy posted at Mudcat. As you've often said, the DOS version works better than any other version we've had. I have a (bootleg) 2005 Digital Tradition in a DOS window in Windows XP, and it opens with two keystrokes and works like a charm. It even beeps out the tunes, which didn't happen in Windows in my previous computers. It took me about ten minutes to install the 2002, and just a little longer to modify it to run the 2005 file.

Has anybody tried the DOS DT on a Beta version of Windows Vista? I talked wiht a Vistar user last night, and he said he has had trouble running some programs on it. I think that the DOS version of the DT is simple enough that it shouldn't cause problems for Vista.
-Joe-


06 Jan 07 - 06:22 PM (#1928654)
Subject: RE: Digital Tradition Upgrade?
From: Captain Ginger

Nice idea - and with Bootcamp for the Mac, a DOS programme is pretty universal.
There's a part of me, though, that would love to see the elegance of Harmony assistant incorporated within a Jave app.
But, trouble is, I know diddly-squit about programming.


06 Jan 07 - 10:56 PM (#1928846)
Subject: RE: Digital Tradition Upgrade?
From: The Fooles Troupe

There are various approaches to running Windows progs now under Linux - I'm not an expert on this - including WINE, etc.

If there is a simple way of running simple DOS progs under Linux, that at least gets around the 'Linux-compatibility' issue - we would only then need the DOS version to get immediate Linux compatibility.

I suggest that most of those who have abandonded 'The Windoze Path of Enlightment' for Linux probably have enough basic 'technical nouse' to cope with getting the thing running with only basic instructions.

~~~~~~~~
I had thought of a 'Wiki-style' Approach with limited access to editing facilities - and I believe that you can 'queue' changes for 'moderation review' before permanent commital.

How difficult would it be to have the 'Wiki thing' implemented as a complete (downloaded and installed) 'standalone package' on each user's PC? (the nightmare of differing OSes rears its head...)

~~~~~~~~~
"Dick...would it be possible or useful if the lyrics database were totally separate from the tunes?"
"Dick, in the meantime, I wonder if it would be worthwhile to go back to issuing a DOS version of the Digital Tradition on a regular basis, perhaps along with an HTML copy posted at Mudcat."

Personal opinions...

1) a web accessible format, especially if kept moderately 'up to date' would serve most users, even those of us on 'slow' dialup, to a degree of usefulness - this might be easiest to implement and update (as per the current format) in a pure HTML style.

2) Re: "It's easy enough to set up a program that will let one find lyrics by title--it's also a fairly useless exercise. Titles are an awful way to search for things. Full text and keyword searches are, in Susan's and my opinion, a necessity. So is the requirement of providing synchronized words and music--many of our users don't read music (either dots or abc), and realizing how the tune fits the words is by no means always obvious. These aren't frills--they're an integral part of the DT's concept. "

I suggest a seperation of the 'basic' and 'all singing, all dancing' formats - while I can see where Dick is coming from, and why, a workable version without the "synchronized words and music" would be easier to get started on and keep updated, and thus more 'useful' (even if not as useful to those with those limited musical technical abilities that he mentions) overall to a fair number of people.

If both versions use the same file formats etc, and the 'deluxe' one can cope with 'missing' (i.e unimplemented or unentered bits of the 'bouncing ball' format stuff) bits, then we have

a) a working more regularly updated basic system

b) an advanced 'deluxe' system that will gradually build up as more items are 'upgraded' (animated bouncing balls, etc) over time.

As per Andrez's suggestions re 'treating it as a real world IT development project' - if we just all sit and wait for the 'all singing, all dancing' Showboat to arrive before we do any implementation, then we won't have much that is useful for some time is my polite and respectful thought...

This suggested breakup of concepts helps to allow several independent teams to be working on related but seperated sections simultaenously.


07 Jan 07 - 12:23 PM (#1929244)
Subject: RE: Digital Tradition Upgrade?
From: dick greenhaus

The problem with tunes (display and playing) is not just with new additions--if we use a new play program, ALL the tunes will have to be converted before they become available.The direction I'm working in now is an improved search program (similar to the old DOS one) which can operate independently of platform; it will have the capacity to link to a music display/play program assuming one can be found or developed.
It's possible that the next edition, then, will be lyrics-only. What are people's feelings about that?


07 Jan 07 - 03:11 PM (#1929421)
Subject: RE: Digital Tradition Upgrade?
From: MMario

Better then nada


07 Jan 07 - 05:11 PM (#1929523)
Subject: RE: Digital Tradition Upgrade?
From: DMcG

I have Harmony Assistant. It includes a built-in programming language that can be used for all kinds of things and I'm pretty confident that I could write a macro for it that could import lots of ABC files and convert them to the Myriad standard if we had to. As has been noted above, these could be made available as an online version, and the plug-in does support a marker showing which note is playing. There is an example at the Myriad Website. Of course, one disadvantage of this is that everyone would have to install the plugin.


08 Jan 07 - 07:43 AM (#1930074)
Subject: RE: Digital Tradition Upgrade?
From: DMcG

Just confirming the above. I've written and run the script and converted a few hundred ABCs as a test.


08 Jan 07 - 08:09 AM (#1930099)
Subject: RE: Digital Tradition Upgrade?
From: The Fooles Troupe

"Better then nada"

As long as it works on my pile of junk, I'll be happy....

:-)


08 Jan 07 - 07:58 PM (#1930866)
Subject: RE: Digital Tradition Upgrade?
From: GUEST,riverboat annie

well, Dick...I'm okay with lyrics only because I can find fiddle tunes so easily on JC's ABC. The folk stuff tunes are easily had as well...


08 Jan 07 - 09:14 PM (#1930926)
Subject: RE: Digital Tradition Upgrade?
From: Ferrara

I would strongly prefer that this project start by concentrating on upgrading the on-line Digitrad database. It is a more feasible, clearly defined, and doable project and I think it would have the biggest payoff for the largest number of people. The programming would be a fraction of what the stand-alone will take. And there is no shortage of qualified (and eager) volunteers to help with the content.

And I really don't think it would be too hard to come up with procedures that ensure Dick and Susan can validate changes and additions done by a picked set of Dick-and-Susan-clones.

I would love to see it. I could live with a lyrics-only version. What I would prefer though, is if a good tune file (midi or mp3) is available through Mudcat, there could be a link to it. Maybe have space for "at most" one midi and one mp3. If they are optional, then music files could be added as they become available but not having them won't keep a song out of Digitrad.

Can't speak for anyone else, but for me about half my use of Digitrad is to get complete lyrics for a song whose tune is known to me. (Most of the other half is searching for songs that fit an Open Sing topic. Love the search capability.)

Rita F


08 Jan 07 - 10:19 PM (#1930991)
Subject: RE: Digital Tradition Upgrade?
From: Linda Goodman Zebooker

I too use the Digitrad to search for Open Sing topic songs or just to create a bigger selection for something like the Getaway. I have a much, much more limited knowledge of songs, though, and depend mightily on hearing the tune, especially when timed visually to the notes. I actually use the downloaded 2002 Digitrad when I'm home, so I can change keys, tempi and sometimes the instrument. I've learned a lot of songs cold, just from the Digitrad, without hearing them anywhere else. I'd be lost without the music. A treasured resource, however it works out.

Linda


08 Jan 07 - 10:22 PM (#1930995)
Subject: RE: Digital Tradition Upgrade?
From: dick greenhaus

Believe it or not, content isn't the problem. And the thought of having different databases for online and stand-alone makes Susan and I shudder to the extent that I fear for the integrity of our home. The search capability of the online version is, IMO, pathetic.
    It's NOT simpler to just update the online version; it's much more difficult than it is to create a new stand-alone one, and then convert that to HTML. The problem of platform independence is one that we're close to licking. The tune handling is not.
The online DT is really a bastardized version of the real DT; created solely because, in the early days, Max had lots of storage space but had problems with processing time due to limited bandwidth. This, hopefully, will not be a limitation in the future.

If the object is simply to increase the number of lyrics available, it's not really necessary top do anything--the forum search can take care of that. I'd hate to see us degrade the product even more than it has been.


09 Jan 07 - 03:19 AM (#1931136)
Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon

Dick, there are different ways of going about things and I'm not saying this is the right choice but in relation to what I've suggested, and in what we went through before, your comments aren't making any sense.

Believe it or not, content isn't the problem.

I'm not convinced it's not a problem but we will see when the new version is out.

And the thought of having different databases for online and stand-alone makes Susan and I shudder to the extent that I fear for the integrity of our home.

That is precicesly what you have now. The suggestion is to reduce this to one but to update the online and forget your stand alone one. This, should you wish, makes it easy to have a trused group of editors - a reduction in work load, not an increase.

The search capability of the online version is, IMO, pathetic.

That could be improved.

It's NOT simpler to just update the online version; it's much more difficult than it is to create a new stand-alone one, and then convert that to HTML.

Takes me about 30 seconds for me to generate a separate html page for each song, cross ref links to keywords, etc. It is not difficult from an SQL database.

The online DT is really a bastardized version of the real DT; created solely because, in the early days, Max had lots of storage space but had problems with processing time due to limited bandwidth. This, hopefully, will not be a limitation in the future.

Please explain.


09 Jan 07 - 01:54 PM (#1931583)
Subject: RE: Digital Tradition Upgrade?
From: GUEST

There seems to be a bit of confusion regarding the various versions of the DT that are extant. The HTML version on the Mudcat site is not created independently--it's derived from a single master version which is also the basis for the Windows and Mac versions.
Yes, one generate a separate HTML page for each song--at 30 seconds per, that would take some 80 hours of steady work. The result would be pretty much what we have now--a database with a lot of useful information that's hard to find. It would still fail to solve the music playing problems that have been a constant nuisance.

When I say that content isn't a problem, it's just what I mean. We now have another 1000 lyrics ready to go, and finding more is no problem.

we're concentrating on the standalone version for a couple of important reasons: a)it's a much better product. b)once the standalone is ready, converting to a web-based database is trivial c) thae standalone is much easier to maintain--just try tracking down duplications of songs in the HTML version.


09 Jan 07 - 03:22 PM (#1931658)
Subject: RE: Digital Tradition Upgrade?
From: Bill D

(I assume that's you, Dick..*smile)

question....when you say you have "another 1000 lyrics ready to go", does that include changes & corrections that folks have fretted about pretty regularly? Do you still mean that changes and updates would only be a once or twice a year happening? It would mean a lot to have the possibility of correcting errors regularly (read that as "almost immediately").


09 Jan 07 - 04:29 PM (#1931716)
Subject: RE: Digital Tradition Upgrade?
From: Susan of DT

Yes, that was Dick at 1:54. I don't know why his cookie wasn't functional.

There has to be one master copy from which new versions, other versions, etc. are generated. Once you have some changes made to copy a and others made to copy b, you have chaos and no way to get them back into one place. The posted version cannot be changed on the fly. Changes go into the master from which the next version is produced.


09 Jan 07 - 05:46 PM (#1931783)
Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon

No Dick, I am aware of the situation with the databases and I meant the run to produce all this takes about 30 seconds. I am aware that doesn't answer the search problem but I didn't suggest it did, I was just pointing out what you said was wrong.

Anyway, I will leave it at this, I've tried to point out some things you have missunderstood but I've no wish to fall out with you or come over as if I'm dictating which way you should go with it.


10 Jan 07 - 01:10 PM (#1932480)
Subject: RE: Digital Tradition Upgrade?
From: dick greenhaus

Guest Jon-
Just out of curiousity, what did you use as a starting point for those HTML pages?


10 Jan 07 - 01:44 PM (#1932520)
Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon

Dick, I'd already built a MySQL datbase from a text file imported from AskSam for this we talked a little about in the last thread on the dt.

I just ran a php script on the MySQL database to produce the HTML version. My coding isn't particularly good but here it is if you want to see how it was made from the database.

function microtime_float()
        {
        list($usec, $sec) = explode(" ", microtime());
        return (float) $usec + (float) $sec;
        }
        
function addkeyword($SongID)
    {
    global $file, $nl;
    $SQL ="Select keyword.KeywordID, keyword.Keword from keyword, songkeyword
    where keyword.KeywordID = songkeyword.KeywordID and songkeyword.SongID = " . $SongID;
    $result = mysql_query($SQL);
    $rs = mysql_fetch_array($result);
    $added = false;
    while (!($rs==0))
        {
        if ($added)
            fwrite($file, ", ");
        else
            $added = true;
        fwrite($file, "<a href='../keyword/" . $rs["Keword"] . ".html'>" . $rs["Keword"] . "</a>");
        $rs=mysql_fetch_array($result);
        }
    fwrite($file, $nl);   
    }


function addtune($SongID)
    {
    global $file, $nl;
    $SQL = "Select ABC, Title From tune, songtune where SongID = " . $SongID . " and songtune.tuneid=tune.tuneid";
    $result = mysql_query($SQL);
    $rs = mysql_fetch_array($result);
    while (!($rs==0))
        {
        fwrite($file, "<a href='../tune/" . $rs["Title"] . ".html'>" .$rs["Title"] . "</a>" . " ");
        $rs=mysql_fetch_array($result);
        }
    fwrite($file, $nl);
    }

function addkeywordsong($KeywordID)
    {
    global $file, $nl;
    $SQL = "Select Title, song.Filename From song, songkeyword
            where song.SongID=songkeyword.SongID and
            KeywordID = " . $KeywordID;
    $result = mysql_query($SQL);
    $rs = mysql_fetch_array($result);
    while (!($rs==0))
        {
        fwrite($file, "<a href='../song/" . $rs["Filename"] . ".html'>" .$rs["Title"] . "</a>" . $nl);
        $rs=mysql_fetch_array($result);
        }
    fwrite($file, $nl);
    }



require("config.php");
require("top.php");

$start = microtime_float();
$nl = "<br>" . chr(13).chr(10);

//add the songs
echo "adding songs<br>";
$SQL = "SELECT * FROM song ORDER BY Title";
$result = mysql_query($SQL);
$rs = mysql_fetch_array($result);
@mkdir("html");
@mkdir("html/song");
@mkdir("html/tune");
@mkdir("html/keyword");
while (!($rs==0))
        {
        $filename = "html/song/" . $rs["Filename"] . ".html";
        $file = fopen($filename, 'w');
        //echo $filename . "<br>";
        $SongID = $rs["SongID"];
        $Song = $rs["Song"];
        fwrite($file, "<b>" . $rs["Title"] . "</b>" . $nl);
        fwrite($file, $Song);
        fwrite($file, "<b>Keywords: </b>");
        addkeyword($SongID);
        fwrite($file, "<b>Tune: </b>");
        addtune($SongID);
        fclose($file);
        $rs=mysql_fetch_array($result);
        }

//add the tunes
echo "adding tunes <br>";
$SQL = "SELECT * FROM tune";
$result = mysql_query($SQL);
$rs = mysql_fetch_array($result);
while (!($rs==0))
        {
        $filename = "html/tune/" . $rs["Title"] . ".html";
        $file = fopen($filename, 'w');
        fwrite($file, str_replace(chr(13), $nl, $rs["ABC"]));
        fclose($file);
        $rs=mysql_fetch_array($result);
        }
        
//add the kewords
echo "adding keywords<br>";
$SQL = "SELECT * FROM keyword";
$result = mysql_query($SQL);
$rs = mysql_fetch_array($result);
while (!($rs==0))
        {
        $filename = "html/keyword/" . $rs["Keword"] . ".html";
        //echo $filename . "<br>";
        $file = fopen($filename, 'w');
        addkeywordsong($rs[KeywordID]);
        fclose($file);
        $rs=mysql_fetch_array($result);
        }

//make an index
echo "adding index<br>";
$indexfilename = "html/index.html";
$indexfile = fopen($indexfilename, 'w');

for ($I=65; $I<=90; $I++)
    {
    fwrite($indexfile, "<a href=".chr($I). ".html>" . chr($I) . "</a> ");
    if ($I == 65)
           $where = "Title <='B'";
    elseif ($I == 90)
       $where = "Title >='Z'";
    else
       $where = "Title >='" . chr($I) . "' AND Title <'" . chr($I+1) . "'";
    $SQL = "SELECT Title, Filename FROM song Where " . $where;
    $result = mysql_query($SQL);
    echo mysql_error();
    $rs = mysql_fetch_array($result);
    $file=fopen("html/" . chr($I) . ".html", 'w');
   while (!($rs==0))
        {
       fwrite($file, "<a href='song/" . $rs["Filename"] . ".html'>" .$rs["Title"] . "<a>" . $nl);
        $rs=mysql_fetch_array($result);
        }
        fclose($file);
    }
fclose($indexfile);
$end = microtime_float();$end = microtime_float();
echo "File built in " . round($end - $start,2) . "seconds";
?>


10 Jan 07 - 04:45 PM (#1932662)
Subject: RE: Digital Tradition Upgrade?
From: Ferrara

Dick, I was suggesting making the online database the master file. The data in each new stand-alone release would be automatically generated as a "snapshot" of the on-line database at specified intervals, making new releases automatic once the basic code for the stand-alone database is in place.

I no longer have the skills to do any of this though.... However I know the Folklore Society can produce reports of its online data in machine-readable format that can be imported into various software.

Is this not possible for the online Digitrad database as well? Since the stand-alone version is being revamped, I would expect that whatever database engine is used would include an import feature? Lots of data of course but not a maintenance nightmare.

And, since the stand-alone database is released in specific versions, there would be no data concurrency problem. The stand-alone version would be a snapshot of the master database at a given time.

Last, about providing linked music files. I would be against having just lyrics and no tunes in the DT. That would be a huge step backward.

But I can't see any rationale for withholding a song from the DT because there is no tune available at present! What is the problem with having some songs without tune files until tunes can be made avaiable? Bill has a saying: "Fifty percent of something is better than 100 percent of nothing."

Rita F


10 Jan 07 - 10:03 PM (#1932830)
Subject: RE: Digital Tradition Upgrade?
From: Malcolm Douglas

And, of course, many DT texts are copied from sources that didn't include tunes in the first place; some of these are 'Child' ballads for which there is no known melody in any case. My personal feeling is that it is better to provide no melody rather than provide one which may never have been used with the text in question, or which is not properly identified. DT files frequently link to unnamed, unidentified tunes that may be anything at all; no information is provided.

Were all those unidentified tunes really sent in without source information? Not the ones I provided (several hundred, I think), though the essential contextual info I included has frequently been omitted or misquoted, both in the DT and in the Mudcat midi annexe.

I agree with Ferrara. The source database needs to be an online version, converted to an easily updateable form in a current format (SQL rather than the obsolete AskSam). That isn't hard to do.

Once there is a reliable central resource into which corrections can be entered as they arise (instead of five or more years too late) standalone versions can be built from it at whatever intervals best suit. Trying to do it the other way round just isn't going to work any more.


11 Jan 07 - 12:14 AM (#1932915)
Subject: RE: Digital Tradition Upgrade?
From: Ferrara

The model for having the online version be the master file is the basis of the concept of "distributed processing." The definitive version of an organization's DB is on a central mainframe but snapshots can be downloaded either at regular intervals or at need. Then they can be used locally in various ways.

Using this model for Digitrad, the snapshots would be downloaded into a new stand-alone database, by an automated process, at regular intervals. And distributed by whatever method you like.

The on-line version would be the only place where changes are applied, and there would be some kind of clear, managed process for applying and reviewing changes.

Rita F


11 Jan 07 - 12:21 AM (#1932921)
Subject: RE: Digital Tradition Upgrade?
From: The Fooles Troupe

I regret not having stated clearer the same basic concept as Rita...


11 Jan 07 - 04:54 AM (#1933018)
Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon

Malcolm, AskSam isn't obsolete. I belive it's good at what it does too but I would favour an SQL relational database over the proprietry flat file format.


11 Jan 07 - 05:08 AM (#1933024)
Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon

Dick, out of curiosity, what problems do you have with my online search? It allows a full text search (natural) as well as matching words or a phrase, allows searches on dt keywords (I called them categories, and youenter them withouy the @) and a combination of both.


11 Jan 07 - 10:13 AM (#1933307)
Subject: RE: Digital Tradition Upgrade?
From: Tom Hamilton frae Saltcoats Scotland

When I go to the digital keyword search and then click one of the boxes I get an error message, I mean it's been like that for months, since last year.

I even click the two boxes at the top of the page above the box that says search


11 Jan 07 - 11:33 AM (#1933386)
Subject: RE: Digital Tradition Upgrade?
From: dick greenhaus

Guest John-
Which online search do you refer to? The one at Mudcat now a)doesn't recaognize phrases containing common articles, so that users keep reporting that they can't find songs. b)doesn't accept wildcards, so that variant spellings present problems c)can't deal withh Boolean relationships.

Malcolm-As long as DigiTrad accepts contributions from anyone, there's no way og guaranteeing "authenticity". Criticizing it for that lack is somewhat like criticizing the Roud index for not including lyrics. Or criticizing a camel for being a poor cow. In retrospect, I feel that we might have been better off not soliciting contributions and keeping a tighter control over what's in there, but if we had it would be a much more limited resource.

Rita-The delays in updating have nothing to do with which version is considered the master copy; it's been due to the several layers of volunteer programmers between us and the online version. What I'm working on now an arrangement that permits me to interact directly with the website, while enabling me to maintain a single platform-independent version. Which will cerainly include the tunes, in some form or other.


11 Jan 07 - 01:28 PM (#1933471)
Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon

I was reffering to mine here Dick. It doesn't meet all your oobjections to the Mudcat one in one search method but get closer.

I'm curious about:

"What I'm working on now an arrangement that permits me to interact directly with the website,"

Please expand on this.


12 Jan 07 - 10:33 AM (#1934352)
Subject: RE: Digital Tradition Upgrade?
From: dick greenhaus

Guest Jon-
I'm very impressed with what you've done. It's certainly a better implementation than the one we currently have on Mudcat.
Could we discuss this offline? My Email URL is dick@camscomusic.com; my phone number (US & Canada) is 800/548-FOLK (3655)


12 Jan 07 - 10:42 AM (#1934366)
Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon

Certainly Dick, I've emailed you to make contact. (I won't be using the phone btw, I'm in the UK).


14 Jan 07 - 07:41 PM (#1936714)
Subject: RE: Digital Tradition Upgrade?
From: Mick Pearce (MCP)

I've mentioned my approach to the design before, but I'll give a description of what I've been doing with the DT.

I still think the way forward is to use a relational database with the semantic structure of the song files. In my database I have tables for:

  Songs - with titles and identifier fields (Child, Laws, Roud, DT#)

  Keywords for Songs - this links the Song table to all the keywords for the song

  SongLines - the entire DT data, but with type added to each line eg Title, Song Text, Notes etc (I think I have 12 types)

  Tunes for Songs - this links the song table to all the tunes given for it

  Tunes - the individual tune names with links to the abc

  Abc - the abc lines for each tune.


Both the text data and the tune data were generated programmatically from the original DT files (in fact I think it's not the last release I've got loaded but the one before). There was a bit of manual processing aferwards (mostly when the heuristics for separating song notes from song lines in the DT failed. I added a switch button to my viewer so that I could go through the files and have selected lines switched from text to notes or vice versa. There were also a handful of multivoice SongWright files that I didn't convert to abc - the extra programming to handle it didn't seem worth it for the few files affected).


I've written a Java browser to access the database (I'm using Java for intended platform independence). The browser part displays all the titles in a tree (title name under start letter nodes) and I make use of the line type information when displaying selected songs - I keep the song notes separate from the song text. I can display the score (I'm currently using abcm2ps to generate the score and I've got a version of the Java postscript interpreter Toastscript that I've
modified to allow it to display and print the score from within the browser. I may automatically display the music in the browser later.

I have implemented a simple karaoke midi player in the browser that displays the song-syllables along with the tune, one line at a time. The midi is generated using abc2midi, so you can see I'm not totally Java yet! My personal opinion is that you don't necessarily need the karaoke to happen on the score the way the current DT browser does. While that's nice, it's a bugger to do unless you're implementing the score writer yourself (which I'm not, certainly not for now; even
later I'd prefer to generate postscript and forgo bouncing ball/highlighting the score), and I'm of the opinion that if you can read the music you don't need the karaoke and if you use the karaoke
you don't need the score.

For searching I allow searches against the following items:
  Identifier: Child, Laws, Roud, DT, exact titles, exact keywords
  Substrings of Titles or keywords
  Substrings of single lines of song text or song's notes
  Substrings in any line of a song text, any line of a song's notes, any line as in the original DT for the song

The search item can be a list of phrases separated by && (for and) or || (for or), each phrase optionally starting with ~ to negate the test (match-> no match, string in item -> string not in item). It's not particularly elegant, but I chose this because in Java it's almost trivial to split the individual items out. At present I can only apply the tests to a single item at a time - eg I couldn't seach for (Child=1) AND ('Nic Jones' In Song Notes). (Though I return the search result titles as a node on the Browse tree, so I could in principle apply a new search to songs only within a node - I don't at the moment, but it shouldn't be too hard to do. I do allow the results of several searches to be added to the result node or clear out the old entries first). At present I force all '&&' or all '||' to be used in a test, but that's because I couldn't be bothered handling parentheses to set the precedence of (A&&B)||C versus A&&(B||C). I could allow both now if I was willing to accept the default operator precedence, but choose not to. In theory there's no limit to how many operators can appear, but some of the whole song searches could take a long time with a lot of conditions. (All theses searches essentially only generate one of three types of queries - a simple one against the Song table, a simple join of Song with Keywords or Song with the DT Lines, a sequence of joins of Song with DTLines linked by INTERSECT or UNION). Also the conjunctions are not order dependent William && Mary would find 'William loves Mary' and 'Mary loves William' (though it would be trivial to allow something like 'William > Mary' to pick out 'William loves Mary' by creating a simple LIKE/regexp condition in the generated search; in fact I like that idea so much I might do it!. I'm not in favour of general regexp expressions being allowed though - it's too easy for people to get them wrong!).

So examples of searches I can do are:


Child#:       Child#: 4||8       finds all entries marked as Child #4 or Child#8
Title:         William && ~Mary   finds song titles containing William but not Mary
Song line:      Dilston && ancient find songs with both of these words in the same line
All song lines: Dilston && ancient finds songs with 'Dilston' in any line of the song and 'ancient' in any line of the song
(the last will find DERWENTWATER'S FAREWELL, the previous wont find anything)

Text searches are currently case insensitive, but the code generating the SQL query can already generate a case sensitive search if I ever add that as an option in the search strings.

This search set can run everything that my Windows DT version can do sslighly faster on some of the searches, but essentially the same order of magnitude), but can run the more complex searches too. Note that in the substring searches I search for any substring, not whole words. So 'Hall' will match 'shall' and 'halleluja' too. The database I'm using (Apache Derby - a pure Java database) doesn't support any whole word text searches . I could index all (non-trivial) words in the database and use those, but I don't find the absence of such searches a problem. There are relational databases that support document handling better than this - Oracle, SQL server and MySQL have support for word and phrase searches (I can't find phrases that span two lines for example) and I may try to do this with my MySQL version of the database (the Java code can access this just by selecting a different database driver, so I shouldn't need to change anything else to run my current version - I currently have the data in Apache Derby, MySQL and MS Access, and could run the browser from any of them - I could also have it as a client to a server of my Derby DB - though I'm usually running on Derby, part of my heading towards an all Java system. I wanted to develop a system that used only free components and for relational databases both MySQL and Derby are free and I can use either from Java, Derby just has a smaller footprint - about 2Mb for the code.)

I haven't added the editor form yet, but that's essentially just putting a lot of fields on a form (I've designed it, just not programmed it) and updating the database from it. My ultimate aim is to use it to add my own songs and tunes to it (I'll just flag DT orignal files separately). I'll also extend the tune info for a song to include links to external Audio files (midi/wav/mp3) and let me be
selected for the song along with the current tunes available.

This is more technical stuff than I'd normally post, but I thought you might be interested.


For the DT I think there are more significant things that can be done to simplify the administration of the system and increase the usefulness of the online database.

Having a proper Add Song form on Mudcat would be my choice for the most important change to the system (moreso than the browser system). At the simplest I'd include Titles, Song Text and Notes plus the Identifier information. It could be stuffed as is into the online database - it needs no more than a flag field to distinguish it from the last released DT.

The searches available could include the songs from the start and the editors could easily locate the new songs for tidying up and possible aapproval for inclusion as release songs (or even deletion - keep an archive note - "Song entered but duplicate of...". It would then be trivial to issue updates to the distributed versions of the DT. The song could remain 'open' for annotations - where people could post possible corrections to the song. These could be displayed along with the notes when displaying the song and again the editors could review these when preparing a new release (or just ongoing - you'll always know when there are open annotations) and either alter the song or leave them as notes. I'd do the same for tunes too.


But as I've said before whatever way the DT goes is fine with me. It's a great resource and it's been provided free by people who put in the time and effort just for the love of it. While I've written my own browser (partly just to get the hang of Java, partly to get some things the way I'd prefer them, and still work in progress, though I have found Java very quick and easy for development - although the browser's not finished, I've probably not spent more than a week on creating it), the (Windows) version that came with the DT is fine (well it's annoying having a new window open for each song you look at - that's one change I did make: the song select/search result tree is on the same page as the song display, I only open a new window for the score view/print). And despite the disproportionate space writing about my browser and its searches compared to my ideas for the online system, it's that last one that I'd really like to have considered.

Mick


14 Jan 07 - 08:08 PM (#1936747)
Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon

Mick, you would probalby take things a lot further than I would. Still think a Mudcat led soulution is the ideal though.... ~(Or at least a no I can't/won't from Max would be helpful).


14 Jan 07 - 08:12 PM (#1936751)
Subject: RE: Digital Tradition Upgrade?
From: GUEST,Jon

(more helpful than a "no man's land that is". I believe they could do it...


14 Jan 07 - 11:40 PM (#1936878)
Subject: RE: Digital Tradition Upgrade?
From: Linda Goodman Zebooker

I appreciate reading these technical postings. I can't begin to understand them, but I can get a small sense of what's underneath something like the digitrad - it's less of an opaque mystery to me now.
Linda


15 Jan 07 - 01:11 AM (#1936921)
Subject: RE: Digital Tradition Upgrade?
From: The Fooles Troupe

MCP

You seem to have done a lot of useful stuff - it also would port to Linux easily, as far as I can tell.

With a separate DB each user could add and sort their own compositions.

Robin


23 Jan 07 - 07:14 AM (#1945295)
Subject: RE: Digital Tradition Upgrade?
From: The Fooles Troupe

FYI: (those interested only)

The ABC Plus Project         

here at sourceforge

has source and binaries for various ABC related things WIN exes & Linux rpms for ABC Plus - which was an intended upgrade to allow harmony as well as the original intended melody that ABC was designed for.

Making Music with ABC Plus
Some ABC Plus choral music
An ABC package for LaTeX
abcm2ps binaries
abcm2ps extensions
abcMIDI binaries
tclabc binaries
NoteEdit
ABCEdit
JedABC
abcpp
abc2prt

Of interest is NoteEdit: a free, great KDE (Linux) program for editing music visually. It imports MIDI and exports in MusiXTeX, Lilypond, PMX, MIDI, and now ABC Plus too! Its original author, Joerg Anders, wrote the ABC Plus support in an amazingly short time after my suggestion/request. Kudos to Joerg!

Recently, Joerg stopped developing NoteEdit, but other developers have taken over: http://noteedit.berlios.de/. Thanks a lot, folks!

also

ABCEdit is a Windows program that integrates abcm2ps, abc2midi and GhostView in a single package. Very easy to use, and it's freeware! Kudos to Joop Coolegem, .

ABCEdit home page is http://www.abcedit.tk/.

Also

JedABC is an extension to the Jed editor that turns it into a powerful and easy to use IDE (Integrated Development Environment) for ABC files, with full integration with preprocessors, converters, players and previewers. It's similar in concept to BarFly or runabc, with several advantages.

JedABC helps you write ABC files with useful features:

All of these are close to what has been claimed is wanted.

There seems to be no 'words' or 'bouncing balls' as yet, but if approached by the Digitrad Development Team, I am sure that the editors/maintainers of these programs would give thought to those extensions.


Robin


19 Sep 07 - 09:02 AM (#2152587)
Subject: RE: Digital Tradition Upgrade?
From: Andrez

Are we there yet?

Is there any chance of a cross platform beta version of the DT with cool text search capabilities any closer to existence?

Would it be possible to have an update on progress, such as it may be since this thread was last activated last January? If there has been some progress I'd be interested to know what if any of the suggestions made in this thread or the others referenced by guest Jon: Tech: Digital Tradition Programmer Needed, been taken on board and developed? Has there been any other behind the scenes progress?

Is there anything anyone can do to help progress the issue: skills, time, money? If skills or time is still an issue could we take up a collection? What would it cost to buy the skills and time? I'd be happy to chuck in a hundred bucks to start off a Digitrad development fund.

Cheers,

Andre

PS: Would it be possible to have a permanent thread where progress notes on the new Digitrad can be posted every now and again?


13 Oct 07 - 01:43 PM (#2170377)
Subject: RE: Digital Tradition Upgrade?
From: dick greenhaus

We're very close to having a platform-independent version of the DT with vastly improved search functions. It includes the capability of frequent single-file updating, addition and deletion, which means that "model years" will be a thing of the past.

We haven't solved the tune problem, however. What's needed is a program that will accept SongWright or ABC files (with lyrics), and play them (preferably with a synchronized "bouncing ball" or some such place marker). Preferably (though not an absolute necessity) it should permit display and printing of the score ("dots".) This program could be integral with an on-line DT, or could be a downloadable program to be run at the user's computer, with the DT providing the input tune data only (this would reduce bandwidth requirements at Mudcat's server considerably.)

I know that some of you have worked on this aspect, with varying degrees of success. I'd greatly appreciate hearing about the specifics of what you've done, and how we can adapt it. Remember, please, that the bulk of the users of DigiTrad are not programmers; it's important to make any such program transparent to the end user, who'd really like to click on a link on the lyrics page to hear and see the music.


14 Oct 07 - 11:25 AM (#2170867)
Subject: RE: Digital Tradition Upgrade?
From: Susan of DT

refresh


15 Oct 07 - 10:24 AM (#2171535)
Subject: RE: Digital Tradition Upgrade?
From: dick greenhaus

No interest at all? Hmmm.


15 Oct 07 - 10:26 AM (#2171538)
Subject: RE: Digital Tradition Upgrade?
From: MMario

Dick - I think this will need to be kept "up" for a while to allow those with the specialized knowledge see it. If I recall - most of those are not among the heavy users.


15 Oct 07 - 10:56 AM (#2171555)
Subject: RE: Digital Tradition Upgrade?
From: John MacKenzie

I'm trying to lose weight Leo!


15 Oct 07 - 11:29 AM (#2171571)
Subject: RE: Digital Tradition Upgrade?
From: Dan Schatz

Do you think it would help to repost the request with a "Tech" prefix? Maybe it would catch the attention of the more computer savvy amongst us.

Dan Schatz


15 Oct 07 - 11:33 AM (#2171574)
Subject: RE: Digital Tradition Upgrade?
From: Jeri

Dan, the more savvy posters posted in the previous 'Tech' thread to which Jon linked.


15 Oct 07 - 11:35 AM (#2171575)
Subject: RE: Digital Tradition Upgrade?
From: Dan Schatz

Oops. That'll teach me not to read only the bottom bit of a thread.

Dan


15 Oct 07 - 11:40 AM (#2171578)
Subject: RE: Digital Tradition Upgrade?
From: Jeri

It's not just you, Dan. The previous offerings seem to have fallen through the cracks. Personally, I don't know much about music programs, but some in that other thread sure did.


17 Oct 07 - 12:50 AM (#2172736)
Subject: RE: Digital Tradition Upgrade?
From: Joe Offer

You know, I'll betcha somebody would like to know that Jeff put the 2007 online edition of the Digital Tradition is now available here at Mudcat. This includes additions that have been made since 2002, but I don't know all the edition dates. I do know that you can search for Feb07 and find the stuff that came out this year.
There's a lot of manual labor involved in this, to keep our old links to Digital Tradition songs working.
Thank you very much, Jeff - and thanks to Dick and Susan for the work they do constantly, putting together all the information. They've added hundreds of songs.
How many songs in the database now, Dick?
-Joe Offer-


17 Oct 07 - 08:53 AM (#2172912)
Subject: RE: Digital Tradition Upgrade?
From: Susan of DT

Thank you, Joe. We did not even know it was up there. I wonder how long it has been there? Isn't communication wonderful?

Thank you, Jeff!

There are 10,000 songs in the Feb07 edition, vs. 9,000 in the 2002 edition, roughly. We are at work on the next edition. I just cleaned up (for the next edition) the obvious problems from running this list for "Feb07" - capitalization in titles, duplicate files, lack of version number in titles, incomplete files.
    I posted my notice less than an hour after Jeff posted the DT - but since I posted to an existing thread, I guess nobody noticed....
    Good thing MMario started a new thread.
    -Joe Offer-


17 Oct 07 - 09:00 AM (#2172916)
Subject: RE: Digital Tradition Upgrade?
From: MMario

hey! Doesn't this deserve a new thread? Yes! It does!


18 Oct 07 - 05:21 AM (#2173471)
Subject: RE: Digital Tradition Upgrade?
From: Geoff the Duck

The old Tech tread about Digitrad and Programmers has just resurfaced with a comment worth note . Here is a direct link to the posting -
BLICKY.
Quack!
GtD.


18 Oct 07 - 08:49 AM (#2173554)
Subject: RE: Digital Tradition Upgrade?
From: MMario

I just thought the notice desserved a thread of it's own Joe; not trying to trump anyone


21 Oct 07 - 08:27 AM (#2175852)
Subject: RE: Digital Tradition Upgrade?
From: GUEST,Andrez

Great to hear news of progress Dick. It would be easier to keep track of progress and catters contributions if there were a single perma-thread that consolidated the other threads. After no response to my query in Sept I thought the whole issue had been swept into the too hard basket until I saw the Digitrad & Programmers thread briefly revived a few nights ago. Please, please make this thread more accessible and not so missable!

Cheers,

Andrez