[BUG] Option "Include All Subdirectories" can lead to infinite loop in File Reader with SFTP

[is-simon]

Describe the bug
When using the "File Reader" with SFTP, using the "Include All Subdirectories" can lead to an infinite loop recursion.

I thought at first that the "Include All Subdirectories" didn't worked, but in fact it's because the connector enters an infinite loop and never comes back from its first poll.

This is because the listDirectories includes both "." and ".." directories, which should be excluded from the recursion.

Expected behavior
The "." and ".." directories should not be included in the recursion.

Actual behavior
The "." and ".." directories are included in the recursion.

Screenshots
Here's a image of my debbugger while inverstigating this problem, showing the directoryStack field.

Environment (please complete the following information):

• Windows (sftp server unknown)
• OpenJDK 17
• Connect 4.4.1

Workaround(s)
I successfully fixed the problem on my local machine by changing the SftpConnection#listDirectories function from this:

@Override
public List<String> listDirectories(String fromDir) throws Exception {
    List<String> directories = new ArrayList<String>();

    cwd(fromDir);

    @SuppressWarnings("unchecked")
    Vector<ChannelSftp.LsEntry> entries = client.ls(".");

    for (Iterator<ChannelSftp.LsEntry> iter = entries.iterator(); iter.hasNext();) {
        ChannelSftp.LsEntry entry = iter.next();

        if (entry.getAttrs().isDir() || entry.getAttrs().isLink()) {
            directories.add(new SftpFileInfo(fromDir, entry).getAbsolutePath());
        }
    }

    return directories;
}

to this

@Override
public List<String> listDirectories(String fromDir) throws Exception {
    List<String> directories = new ArrayList<String>();

    cwd(fromDir);

    @SuppressWarnings("unchecked")
    Vector<ChannelSftp.LsEntry> entries = client.ls(".");

    for (Iterator<ChannelSftp.LsEntry> iter = entries.iterator(); iter.hasNext();) {
        ChannelSftp.LsEntry entry = iter.next();

        if ((entry.getAttrs().isDir() || entry.getAttrs().isLink()) && !".".equals(entry.getFilename()) && !"..".equals(entry.getFilename())) {
            directories.add(new SftpFileInfo(fromDir, entry).getAbsolutePath());
        }
    }

    return directories;
}

But I'm not sure it is the right way to fix this soo I'll let you decide.

[jdonextgen]

Thanks @is-simon for opening this Issue. We have implemented your fix and it will go out in our 4.5.2 release.