🚀 go-pugleaf

Hi guys,

I'm trying to solve a problem it's giving me an headhache. I'm a
newbie in java lang. I'm developing a web spider using jericho
libraries. My classes take a url, connect to it, put all the unique
links present in the html page in a Hashtable as key, then it takes
all the words of the page and put it as value of each key. What I need
is a recursive system that cycle through the hashtable and put new
found links in the Hashtable so the cycle would notice it and continue
until all the links have been parsed.
This is an example of what I would need

public class Parse {

public static Hashtable linksList = new Hashtable();
public static String url = "http://myaddress.com"

   public static doParse() {
     linksList.put(url,new Hashtable());
  
      //generic cycle to fill the hashtable
      for (Enumeration en = linksList.keys(); en.hasMoreElements(); )
{
   	String urlToParse = en.nextElement();
	parse(htmlText);
      }
   }


   public static parse(String url) {
        String htmlText = Connection.getString(url); //get the html of
a given url
        Hashtable words = getWords(htmlText); //get all the words of a
given html text
        linksList.put(url, words);
        for (... all the links in a fiven htmlText) {
             String newUrl; //each new links found in the given html
text
             linksList.put(newUrl, new Hashtable());
        }


   }
}

As you can see the parse method add recursively new hashtable element
for each new url it found.
The generic cycle should be aware of the change that has been made and
go on until the parse method
won't add any new link. This would let me to fill the hashtable with
an url and the words contained in that
url for all the pages in a particoular site. Then, when my ht would be
completely filled I would add
it to a db, having only one connection to it.

How would I do that? I see that HT doesn't see any changes made to
itself inside a cycle. What other
object I should use? Any other idea on how to perform it? I looked
through the internet but I only found
solution on how to avoid the concurrentModificationException on
ArrayList, not code samples on how to
do it.

I really hope you can help me as I'm stuck with this problem and have
no idea on how to solve it
Thanks a lot in advance!!!

Bye for now!

mario