How to programmatically click through a list of elements if one has to wait for a click to load a set of predefined new elements?

It seems to get to job done, but the Promise will always remain pending

You must call resolve() or reject() (or throw an error) inside the executor function of the Promise, otherwise the Promise will remain pending forever:

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/Promise#description says:

The executor's completion state has limited effect on the promise's state:

• The executor return value is ignored. return statements within the executor merely impact control flow and alter whether a part of the function is executed, but do not have any impact on the promise's fulfillment value. If executor exits and it's impossible for resolveFunc or rejectFunc to be called in the future (for example, there are no async tasks scheduled), then the promise remains pending forever.
• If an error is thrown in the executor, the promise is rejected, unless resolveFunc or rejectFunc has already been called.

Your code has to look like this:

function waitForElement(querySelector, elemArr) {
    return new Promise((resolve, reject) => {
    
        if (elemArr.length) {
            console.log(elemArr);
            elemArr.pop().click();
        } else {
            resolve();
            return;
        }
    
        if (document.querySelectorAll(querySelector).length) {
            waitForElement(querySelector, elemArr).then(() => resolve());
        }
    
        const observer = new MutationObserver(() => {
            if (document.querySelectorAll(querySelector).length) {
                observer.disconnect();
                waitForElement(querySelector, elemArr).then(() => resolve());
            }
        });
      
        observer.observe(document.body, {
            childList: true, 
            subtree: true
        });
    });
}

const t = Array.from(document.querySelectorAll('td.colTitle'));

const initialPromise = waitForElement("audio", t);
initialPromise.then(() => console.log('ready'));

Your code will only resolve the last recursively created Promise if elemArr.length is 0. By adding then(resolve) to the returned Promises, you can recursively resolve the parent Promise until initialPromise is resolved.

I assume that there is a simpler solution (and one that doesn't potentially run out of memory on a page with thousands of episodes)

If you want to keep it simple you can click the "title" elements as you are already doing. Instead of a recursive function you could use something like this:

async function collectUrls() {
    // This div gets filled with the audio player (audio element, title text, ...) if you click on an radio show
    const audioContainer = document.querySelector('#mainAudio');

    // This array keeps track of all audio urls
    const urlList = [];

    var observer = new MutationObserver(() => {
        const audioElement = audioContainer.querySelector('audio');
        // If an audio element exists, the source is added to the url list
        if (audioElement) {
            urlList.push(audioElement.querySelector('source').src);
            document.dispatchEvent(new Event('playerSourceChanged'));
        }
    })
    observer.observe(audioContainer, { subtree: true, childList: true });

    const titleElements = Array.from(document.querySelectorAll('td.colTitle'));
    for (let i = 0; i < titleElements.length; i++) {
        const titleElement = titleElements[i];
        const changePromise = new Promise(resolve => document.addEventListener('playerSourceChanged', () => resolve(), { once: true }));
        titleElement.click();
        // Wait until the mutation observer detects a change
        await changePromise;
        console.log(`${i+1} / ${titleElements.length}`);
        // delay to prevent server errors
        await new Promise(resolve => setTimeout(() => resolve(), 200))
    }

    console.log(urlList);
    copy(urlList.map(x => `"${x}"`).join(' '));
}

collectUrls();

An event is used to pause the for loop after a click until the server provides the audio file. See Event and EventTarget for more information about events.

Alternatively you could execute the onclick function yourself. It basically calls https://www.otrr.org/OTRRLibrary/php/files.php?qid=jukeC&ide=<entry-id>. This seems to prepare the server to provide the mp3 file and returns a json object with some information and the filename. In addition, this method prevents the automatic loading of audio files into the browser and is therefore faster:

async function collectUrls() {
    // This array keeps track of all audio urls
    const urlList = [];

    const titleElementIds = Array.from(document.querySelectorAll('td.colTitle')).map(e => e.parentElement.dataset.ide);

    for (let i = 0; i < titleElementIds.length; i++) {
        const ide = titleElementIds[i];
        const data = await fetch(`https://www.otrr.org/OTRRLibrary/php/files.php?qid=jukeC&ide=${ide}`).then(r => r.json());
        urlList.push(`https://otrr.org/OTRRLibrary/jukebox/${data.file.replaceAll('+', '%20')}`);

        console.log(`${i} / ${titleElementIds.length}`);
    }

    console.log(urlList);
    copy(urlList.map(x => `"${x}"`).join(' '));
}

collectUrls();

posted 11 months ago

CC BY 4.0

Zer0‭

71 reputation 0 2 7 0