top of page
Search
7 min read

Getting Programming help from ChatGPT for creating immersive text on Everything Neurodiversity


One of the ongoing plans I have for this site is to create immersible readers (clickable transcripts of text, with audio that plays when it is clicked)


So I went to the handy new ChatGPT bot and asked for help:




Sure, I'd be happy to help with that. To create HTML code that will display clickable text from a .srt or .vtt file, you can use the <track> element in HTML5. The <track> element allows you to specify timed tracks for media elements, such as video or audio. You can use it to specify a .srt or .vtt file that contains the text that you want to display.


To use the <track> element, you first need to create a <video> or <audio> element to hold the media that you want to play. Then, you can add the <track> element inside the <video> or <audio> element, like this:




<video>
  <track src="your-file.srt" kind="captions" srclang="en" label="English" />
</video>




In this example, the src attribute specifies the location of the .srt or .vtt file that contains the text to display, the kind attribute specifies that the track is a set of captions, and the srclang and label attributes specify the language and label for the track, respectively.


To make the text clickable, you can use JavaScript to add an event listener to the <track> element that listens for the click event. When the user clicks on the text, the event listener can call a function that plays the media and highlights the words as they are spoken.


Here is an example of how you could do this using JavaScript:



// Get the track element
var track = document.querySelector("track");

// Add an event listener to the track element
track.addEventListener("click", function(event) {
  // When the user clicks on the text, call a function to play the media and highlight the words
  playMedia();
});

function playMedia() {
  // Code to play the media and highlight the words goes here
}


Inside the playMedia() function, you can add code to play the media and highlight the words as they are spoken. There are a few different ways you could do this, but one approach is to use the currentTime property of the <video> or <audio> element to keep track of the current time in the media, and the WebVTT.parse method to parse the .srt or .vtt file and get the text and timestamps for each word.


Here is an example of how you could do this:



function playMedia() {
  // Get the video or audio element
  var media = document.querySelector("video");

  // Parse the .srt or .vtt file using the WebVTT.parse method
  WebVTT.parse(track.track.cues, function(cue) {
    // Loop through each word in the file
    for (var i = 0; i < cue.length; i++) {
      // Get the text and start time for the current word
      var text = cue[i].text;
      var startTime = cue[i].startTime;

      // If the current time in the media is greater than or equal to the start time for the current word, highlight the word
      if (media.currentTime >= startTime) {
        // Code to highlight the word

Now I just need to figure out how to implement this on the site and I can add new immersive text (currently done with OtterAI) on the site. The transcriptions will be created with Whisper AI as well. This is something that is already implemented.




 

Ran into a few hiccups with this solutions, mainly surrounding the audio file size. For example a huberman lab podcast can run close to 150mb, So I went back to the handy ChatGPT for some new ideas. -- What if I just embedded the youtube video and created the clickable transcript on the page for the user to interact with? This solution could solve all the problems I'm looking at.





To create a clickable transcript for a YouTube video that is embedded on your page, you'll need to use a combination of HTML, JavaScript, and the YouTube API. Here's a general outline of the steps you'll need to follow:

  1. First, you'll need to create a basic HTML structure for your page, including the <html>, <head>, and <body> tags.

  2. Within the <body> tag, you'll need to add the YouTube video to the page using the <iframe> tag. This tag should include the src attribute, which specifies the URL of the YouTube video you want to embed.

  3. Next, you'll need to add the transcript of the video to the page. This can be done by simply adding the text to the page, with each line of the transcript appearing on its own line in the HTML code.

  4. To make the text on the page clickable, you'll need to wrap each line of the transcript in a <a> (anchor) tag. This tag should include the href attribute, which specifies the time in the video that you want to play when the user clicks on the text. For example, if you want the video to start playing at the 30-second mark when the user clicks on a particular line of the transcript, you would specify #t=30s as the value of the href attribute.

  5. Now, you'll need to add some JavaScript code that will handle the click event on the text and play the YouTube video at the specified time. This code should listen for clicks on the <a> tags, and when a click is detected, it should use the YouTube API to seek to the specified time in the video and start playing the video.

  6. Finally, you'll need to add some additional JavaScript code that will highlight the text on the page as the video plays. This code should use the YouTube API to track the current time of the video, and it should compare this time to the times specified in the href attributes of the <a> tags. If the current time matches the time specified in an <a> tag, the code should highlight the corresponding text on the page.

As you can see, this is a fairly complex task that involves working with several different technologies. If you have any specific questions as you work on implementing this code, feel free to ask and I'll do my best to help. Good luck!







Sample code from ChatGPT: 




<html>
  <head>
    <title>Clickable Transcript Example</title>
  </head>
  <body>
    <!-- Embed the YouTube video using the <iframe> tag -->
    <iframe width="560" height="315" src="https://www.youtube.com/embed/VIDEO_ID" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
    
    <!-- Add the transcript of the video to the page -->
    <p>
      <a href="#t=0s">This is the first line of the transcript</a><br>
      <a href="#t=10s">This is the second line of the transcript</a><br>
      <a href="#t=20s">And so on...</a><br>
    </p>
    
    <!-- Add a <div> element that will be used to display the current time of the video -->
    <div id="current-time"></div>
    
    <!-- Load the YouTube API and add the JavaScript code that will handle the click event on the text and play the video -->
    <script src="https://www.youtube.com/iframe_api"></script>
    <script>
      // Create a global variable that will hold a reference to the YouTube player
      var player;
      
      // This function will be called by the YouTube API when the player is ready
      function onPlayerReady(event) {
        // Listen for clicks on the <a> tags in the transcript
        var links = document.querySelectorAll('a');
        links.forEach(function(link) {
          link.addEventListener('click', onLinkClick);
        });
      }
      
      // This function will be called when a user clicks on a link in the transcript
      function onLinkClick(event) {
        // Get the time specified in the href attribute of the clicked link
        var time = event.target.getAttribute('href').substring(3);
        
        // Seek to the specified time in the video and start playing
        player.seekTo(time);
        player.playVideo();
      }
      
      // This function will be called by the YouTube API every time the video's current time is updated
      function onPlayerStateChange(event) {
        // Get the current time of the video
        var currentTime = player.getCurrentTime();
        
        // Update the <div> element with the current time
        document.getElementById('current-time').innerHTML = 'Current time: ' + currentTime + 's';
        
        // Loop through the <a> tags in the transcript and highlight the text if the current time matches the time specified in the href attribute
        var links = document.querySelectorAll('a');
        links.forEach(function(link) {
          var time = link.getAttribute('href').substring(3);
          if (time == currentTime) {
            link.style.backgroundColor = 'yellow';
          } else {
            link.style.backgroundColor = 'initial';
          }
        });
      }
      
      // This function will be called by the YouTube

of course parsing a transcript into tons of anchors was not something I was looking to do for the amount of video I wanted to cover: 






// Read the .srt file and split it into individual lines
var lines = srtFileString.split('\n');

// Loop through the lines of the .srt file
for (var i = 0; i < lines.length; i++) {
  var line = lines[i];
  
  // Check if the line is a timecode line (e.g. "00:00:01,000 --> 00:00:02,000")
  if (line.match(/\d\d:\d\d:\d\d,\d\d\d --> \d\d:\d\d:\d\d,\d\d\d/)) {
    // Parse the start and end times from the timecode line
    var startTime = line.substring(0, 12);
    var endTime = line.substring(17, 29);
    
    // Read the next line, which should be the text of the transcript
    var text = lines[++i];
    
    // Generate an <a> tag with the start time as the href attribute
    var link = '<a href="#t=' + startTime + '">' + text + '</a><br>';
    
    // Add the link to the page
    document.body.innerHTML += link;
  }
}



of course the sample it gave used JavaScript, i'm more familiar with python and the program to generate the .srt is already in python so I asked the nice chatbot if it could help again:




The python program example: 



# Read the .srt file and split it into individual lines
lines = srt_file_string.split('\n')

# Loop through the lines of the .srt file
for line in lines:
  # Check if the line is a timecode line (e.g. "00:00:01,000 --> 00:00:02,000")
  if re.match(r'\d\d:\d\d:\d\d,\d\d\d --> \d\d:\d\d:\d\d,\d\d\d', line):
    # Parse the start and end times from the timecode line
    start_time = line[0:12]
    end_time = line[17:29]
    
    # Read the next line, which should be the text of the transcript
    text = lines[i+1]
    
    # Generate an <a> tag with the start time as the href attribute
    link = '<a href="#t=' + start_time + '">' + text + '</a><br>'
    
    # Add the link to the page
    print(link)

This code reads the .srt file as a string, splits it into individual lines, and then loops through the lines to extract the timecodes and the corresponding transcript text. For each timecode, it generates an <a> tag with the start time as the href attribute, and then prints the <a> tag to the console.


And finally, the same code to print to a file: 





# Read the .srt file and split it into individual lines
lines = srt_file_string.split('\n')

# Open the file where the <a> tags will be written
with open('output.html', 'w') as f:
  # Loop through the lines of the .srt file
  for line in lines:
    # Check if the line is a timecode line (e.g. "00:00:01,000 --> 00:00:02,000")
    if re.match(r'\d\d:\d\d:\d\d,\d\d\d --> \d\d:\d\d:\d\d,\d\d\d', line):
      # Parse the start and end times from the timecode line
      start_time = line[0:12]
      end_time = line[17:29]

      # Read the next line, which should be the text of the transcript
      text = lines[i+1]

      # Generate an <a> tag with the start time as the href attribute
      link = '<a href="#t=' + start_time + '">' + text + '</a><br>'

      # Write the link to the file
      f.write(link)

Now its time to start testing. Stay tuned.




23 views

Comments

Thank You for Visiting Everything Neurodiversity!

Hello, Thank you for visiting Everything Neurodiversity. This site is a passion project of mine. The hosting costs are minimal and I try to dedicate time to it whenever I can. I intend to keep this site as educational and ad free. 

​

​

I have learned a great deal from working on this site and the social platforms that go along with it. So much that I have started another site dedicated to building a more sustainable and easier fashion shopping expiereince. It has recently been selected for sponsorship in the Microsoft Founders Hub Program and I'm excited for the new developments this will enable. The first 10,000 users who make a purchase through the site will get lifetime Premier Status enabling rewards up to 17%! Check it out here: RunwayRewards.Shop or browse the integrated page below: 

popular posts

music

Books

1
2

HR Resources

events

bottom of page