How deep is the visible area of HtmlAgilityPack?

I need to grab some posts from a blog. All went well until I've wanted to get the post creation date. The DOM-tree for it is:

div class="stories-feed__container" 
  -> article 
     -> div class="story__main" 
       -> div class="story__footer" 
         -> div class="story__user user" 
           -> div class="user__info-item" 
             -> time datetime="date and time in UTC format".

So I wrote the code:

    HtmlWeb web = new HtmlWeb();
    HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
    doc = web.Load("https://pikabu.ru/@serhiy1994");
    string postDate = doc.DocumentNode.SelectSingleNode("//div[contains(@class, 'stories-feed__container')]/article[2]/div[contains(@class, 'story__main')]/div[contains(@class, 'story__footer')]/div[contains(@class, 'story__user user')]/div[contains(@class, 'user__info-item')]/time").GetAttributeValue("datetime", "NULL"); // e.g. for the 2nd article on the page

And it returns the NullReferenceException.
BUT if you stop at the "div class="story__user user"" level (e.g.,

    string postDate = doc.DocumentNode.SelectSingleNode("//div[contains(@class, 'stories-feed__container')]/article[2]/div[contains(@class, 'story__main')]/div[contains(@class, 'story__footer')]/div[contains(@class, 'story__user user')]").InnerHtml;

it works properly and return you the inner HTML-code.
So I think there is something like 'maximum visibility level" for HtmlAgilityPack, and you won't able to manipulate with the deeper markdown.

Am I right or I'm coding something wrong?

The original page code is here: https://pastebin.com/jFC0XD9C

HtmlAgility will scrape the entire website, regardless of how deep you want to go. You can use this to get to the item you are looking for since you dont have to provide the entire path.

This will search the entire site and look for the first <div> tag that has the class name user__info-item. You can also change SelectSingleNode to SelectNodes if there are multiple tags then loop through them to get the dates.

    HtmlWeb web = new HtmlWeb();
    HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
    doc = web.Load("https://pikabu.ru/@serhiy1994");
    var postDate = doc.DocumentNode.SelectSingleNode("//div[@class='user__info-item']/time");
    Console.WriteLine(postDate.InnerText);

Whats wrong with your code?

Reason the code above you have doesnt work is because there is another div that you are missing, '<div class="user__info user__info_left">'.

If you write your code like this, it works.

    var nodes = doc.DocumentNode.SelectSingleNode("//div[@class='story__main']/div[@class='story__footer']/div[@class='story__user user']/div[@class='user__info user__info_left']/div[@class='user__info-item']/time");
    Console.WriteLine(nodes.InnerText);

Another way

Another way to do it is by searching for a parent div. Once you find the parent tag, search under that tag to find what you are looking for.

    var nodes = doc.DocumentNode.SelectNodes("//div[@class='story__user user']");
    foreach (HtmlNode node in nodes)
    {
        // Search within each node using .// notation
        var timeNodes = node.SelectSingleNode(".//div[@class='user__info-item']/time");
        Console.WriteLine(timeNodes.InnerText);
    }

Tested Code here