Public Class Form1 Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load TextBox1.Multiline = True WebBrowser1.Navigate("http://www.youtube.com/results?search_query=hip+hop+samples&search_type=&aq=f") End Sub Private Sub WebBrowser1_DocumentCompleted(ByVal sender As System.Object, ByVal e As System.Windows.Forms.WebBrowserDocumentCompletedEventArgs) Handles WebBrowser1.DocumentCompleted Dim theElementCollection As HtmlElementCollection = WebBrowser1.Document.GetElementsByTagName("a") For Each curElement As HtmlElement In theElementCollection If curElement.GetAttribute("href").Contains("/user/") Then TextBox1.Text += curElement.GetAttribute("innerText") & vbCrLf End If Next End Sub End Class

Best regards,
Martin Xie

You need to be more specific about your request. You think "extract links from web page" is specific? Not really, when you also say you are new to VB and we are not sure how much example code you need.

Have you looked as the page source? Are you only interested in the URL inside of the <Link> tags, or do you want <A> tags?

Here's a start. If you grab a copy of the page source and put it in a file locally, then this snippet might help. This only find the URLs in the href attributes inside of <link> tags. If letter case varies, add the RegexOptions.IgnoreCase.

        Dim tr As TextReader
        tr = File.OpenText("..\..\results.txt")
        Dim test As String = tr.ReadToEnd()
        Dim mx As Match

        Dim pattern As String = "<link[^>]*?href=""(?<link>[^""]*?)"""
        For Each mx In Regex.Matches(test, pattern, RegexOptions.Multiline + RegexOptions.Compiled)
            Console.WriteLine("{0}", mx.Groups("link").Value)
        Next

You need to be more specific about your request. You think "extract links from web page" is specific? Not really, when you also say you are new to VB and we are not sure how much example code you need.

Have you looked as the page source? Are you only interested in the URL inside of the <Link> tags, or do you want <A> tags?

Here's a start. If you grab a copy of the page source and put it in a file locally, then this snippet might help. This only find the URLs in the href attributes inside of <link> tags. If letter case varies, add the RegexOptions.IgnoreCase.

        Dim tr As TextReader

        tr = File.OpenText("..\..\results.txt")

        Dim test As String = tr.ReadToEnd()

        Dim mx As Match



        Dim pattern As String = "<link[^>]*?href=""(?<link>[^""]*?)"""

        For Each mx In Regex.Matches(test, pattern, RegexOptions.Multiline + RegexOptions.Compiled)

            Console.WriteLine("{0}", mx.Groups("link").Value)

        Next

---

Les Potter, Xalnix Corporation, Yet Another C# Blog

Hey Xalnix!,

Thanks for your reply. You're right, I wasn't being very specific.

Here's a more specified inquiry:

How would I extract the USER NAME text of the following page (http://www.youtube.com/results?search_query=hip+hop+samples&search_type=&aq=f) and send onlythe usernames into a textbox1.text???


Even more specified: How would I extract the "YOUTUBE" "USER NAMES" off of this page and send them to a textbox with multiple lines???


Hope this is specific enough and thanks again for your help !!!

Then try this...

        Dim tr As TextReader

        tr = File.OpenText("..\..\results.txt")

        Dim test As String = tr.ReadToEnd()

        Dim mx As Match



        Dim pattern As String = "<a\s+id=""video-from-username[^>]*?href=""(?<link>[^""]*?)""[^>]*>(?<username>.*?(?=</a>))"

        For Each mx In Regex.Matches(test, pattern, RegexOptions.Multiline + RegexOptions.Compiled)

            Console.WriteLine("{0} ({1}, {2})", mx.Value, mx.Groups("link").Value, mx.Groups("username").Value)

            TextBox1.AppendText(mx.Groups("username").Value)

            TextBox1.AppendText(vbCrLf)

        Next

---

Les Potter, Xalnix Corporation, Yet Another C# Blog

Public Class Form1 Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load TextBox1.Multiline = True WebBrowser1.Navigate("http://www.youtube.com/results?search_query=hip+hop+samples&search_type=&aq=f") End Sub Private Sub WebBrowser1_DocumentCompleted(ByVal sender As System.Object, ByVal e As System.Windows.Forms.WebBrowserDocumentCompletedEventArgs) Handles WebBrowser1.DocumentCompleted Dim theElementCollection As HtmlElementCollection = WebBrowser1.Document.GetElementsByTagName("a") For Each curElement As HtmlElement In theElementCollection If curElement.GetAttribute("href").Contains("/user/") Then TextBox1.Text += curElement.GetAttribute("innerText") & vbCrLf End If Next End Sub End Class

Best regards,
Martin Xie

Hi Inforequester,

Since all the hyperlink elementsrelated to Username on the page is in such HTML format:

<a id="video-from-username-kcqPq5189VQ" class="hLink" href="/user/redhooknoodles">redhooknoodles</a>

Thus, you can locate them and retrieve text programmically like this:

Public Class Form1 Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load TextBox1.Multiline = True WebBrowser1.Navigate("http://www.youtube.com/results?search_query=hip+hop+samples&search_type=&aq=f") End Sub Private Sub WebBrowser1_DocumentCompleted(ByVal sender As System.Object, ByVal e As System.Windows.Forms.WebBrowserDocumentCompletedEventArgs) Handles WebBrowser1.DocumentCompleted Dim theElementCollection As HtmlElementCollection = WebBrowser1.Document.GetElementsByTagName("a") For Each curElement As HtmlElement In theElementCollection If curElement.GetAttribute("href").Contains("/user/") Then TextBox1.Text += curElement.GetAttribute("innerText") & vbCrLf End If Next End Sub End Class

Best regards,
Martin Xie

---

Please remember to mark the replies as answers if they help and unmark them if they provide no help.
Welcome to the All-In-One Code Framework! If you have any feedback on our support, please contact msdnmg@microsoft.com

I'm sorry, I must disagree. The pattern extracts "usernames" from your example page. It places those "usernames" in a textbox one on each line. As an added bonus, it prints additional information to the Console (which is only visible in the debugger under the Output window). Since you haven't said where the "username" is located, I've taken a stab at it myself. If you now want both "username" and "account name", you need to ask for that. We can't read minds (thought we often make very good guesses at what folks are thinking).

Now, if I don't understand what you are asking for, and I am the only one even attempting to understand you, you could consider that maybe you are notbeing clear.

If you want both Usernames and Account Names, why don't you copy a few lines of text from your sample and highlight which is which and what parts of the smaller sample you really want to capture.

I am assuming you know VB well enough. If that's not the case, that's fine too. We can work through that.

---

Les Potter, Xalnix Corporation, Yet Another C# Blog

Then try this...

        Dim tr As TextReader

        tr = File.OpenText("..\..\results.txt")

        Dim test As String = tr.ReadToEnd()

        Dim mx As Match



        Dim pattern As String = "<a\s+id=""video-from-username[^>]*?href=""(?<link>[^""]*?)""[^>]*>(?<username>.*?(?=</a>))"

        For Each mx In Regex.Matches(test, pattern, RegexOptions.Multiline + RegexOptions.Compiled)

            Console.WriteLine("{0} ({1}, {2})", mx.Value, mx.Groups("link").Value, mx.Groups("username").Value)

            TextBox1.AppendText(mx.Groups("username").Value)

            TextBox1.AppendText(vbCrLf)

        Next

---

Les Potter, Xalnix Corporation, Yet Another C# Blog

Imports System.IO

Imports System.Text.RegularExpressions

Class Form1


You only said then try this:

Dim tr As TextReader
        tr = File.OpenText("..\..\results.txt")
        Dim test As String = tr.ReadToEnd()
        Dim mx As Match

        Dim pattern As String = "<a\s+id=""video-from-username[^>]*?href=""(?<link>[^""]*?)""[^>]*>(?<username>.*?(?=</a>))"
        For Each mx In Regex.Matches(test, pattern, RegexOptions.Multiline + RegexOptions.Compiled)
            Console.WriteLine("{0} ({1}, {2})", mx.Value, mx.Groups("link").Value, mx.Groups("username").Value)
            TextBox1.AppendText(mx.Groups("username").Value)
            TextBox1.AppendText(vbCrLf)
        Next<br/>

It is ok to disagree as long as we are not disagreeable.

---

John Grove - TFD Group, Senior Software Engineer, EI Division, http://www.tfdg.com

Thank you xalnix for your friendly help!

Hi Inforequester,

You're welcome!

Actually, xalnix didn't misunderstand you. He came up with another solution, that is Regular Expressions,to retrieve all Username text of YouTube page. I test his code sample and it works perfect as mine :)

Prerequisites: Drag&drop WebBrowser1 and TextBox1 onto Form1.

Imports System.IO Imports System.Text.RegularExpressions Public Class Form1 Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load TextBox1.Multiline = True WebBrowser1.Navigate("http://www.youtube.com/results?search_query=hip+hop+samples&search_type=&aq=f") End Sub Private Sub WebBrowser1_DocumentCompleted(ByVal sender As System.Object, ByVal e As System.Windows.Forms.WebBrowserDocumentCompletedEventArgs) Handles WebBrowser1.DocumentCompleted Dim htmlCode As String = WebBrowser1.DocumentText.ToString 'Get html content of web page Dim sw As New StreamWriter(Application.StartupPath & "results.txt") sw.WriteLine(htmlCode) sw.Close() Dim tr As TextReader tr = File.OpenText(Application.StartupPath & "results.txt") Dim test As String = tr.ReadToEnd() Dim mx As Match Dim pattern As String = "<a\s+id=""video-from-username[^>]*?href=""(?<link>[^""]*?)""[^>]*>(?<username>.*?(?=</a>))" For Each mx In Regex.Matches(test, pattern, RegexOptions.Multiline + RegexOptions.Compiled) Console.WriteLine("{0} ({1}, {2})", mx.Value, mx.Groups("link").Value, mx.Groups("username").Value) TextBox1.AppendText(mx.Groups("username").Value) TextBox1.AppendText(vbCrLf) Next End Sub End Class

Tutorial: Introduction to Regular Expressions

http://msdn.microsoft.com/en-us/library/28hw3sce.aspx


Best regards,
Martin Xie

---

Please remember to mark the replies as answers if they help and unmark them if they provide no help.
Welcome to the All-In-One Code Framework! If you have any feedback on our support, please contact msdnmg@microsoft.com

I said mr xalnix didn't understand me because I asked him to be specific with the codesand he only gave me the cake without the icing when he said"

Dim tr As TextReader
tr = File.OpenText("..\..\results.txt")
Dim test As String = tr.ReadToEnd()
Dim mx As Match

Dim pattern As String = "<a\s+id=""video-from-username[^>]*?href=""(?<link>[^""]*?)""[^>]*>(?<username>.*?(?=</a>))"
For Each mx In Regex.Matches(test, pattern, RegexOptions.Multiline + RegexOptions.Compiled)
Console.WriteLine("{0} ({1}, {2})", mx.Value, mx.Groups("link").Value, mx.Groups("username").Value)
TextBox1.AppendText(mx.Groups("username").Value)
TextBox1.AppendText(vbCrLf)
Next
"


This alone will give errors, especailly when your new and need the entire code. :)

You on the other hand knew exactly what it was that I was looking for and wasn't acting like I wasn't being specific enough. Superman to the rescue flying around these forums. LOL :)

Thanks man!!!

I'm sorry, I must disagree. The pattern extracts "usernames" from your example page. It places those "usernames" in a textbox one on each line. As an added bonus, it prints additional information to the Console (which is only visible in the debugger under the Output window). Since you haven't said where the "username" is located, I've taken a stab at it myself. If you now want both "username" and "account name", you need to ask for that. We can't read minds (thought we often make very good guesses at what folks are thinking).

Now, if I don't understand what you are asking for, and I am the only one even attempting to understand you, you could consider that maybe you are notbeing clear.

If you want both Usernames and Account Names, why don't you copy a few lines of text from your sample and highlight which is which and what parts of the smaller sample you really want to capture.

I am assuming you know VB well enough. If that's not the case, that's fine too. We can work through that.

---

Les Potter, Xalnix Corporation, Yet Another C# Blog

Dim tr As TextReader
        tr = File.OpenText("..\..\results.txt")
        Dim test As String = tr.ReadToEnd()
        Dim mx As Match

        Dim pattern As String = "<a\s+id=""video-from-username[^>]*?href=""(?<link>[^""]*?)""[^>]*>(?<username>.*?(?=</a>))"
        For Each mx In Regex.Matches(test, pattern, RegexOptions.Multiline + RegexOptions.Compiled)
            Console.WriteLine("{0} ({1}, {2})", mx.Value, mx.Groups("link").Value, mx.Groups("username").Value)
            TextBox1.AppendText(mx.Groups("username").Value)
            TextBox1.AppendText(vbCrLf)
        Next