Alternative to ParallelForEach That can allow me to kill parallel processes immediately on Application Exit?

I am doing a simple console application that loads files from a database into a hashset. These files are then processed in a parallel foreach loop. This console application does launch a new Process object for each files it needs to process. So it opens new console windows with the application running. I am doing it this way because of logging issues I have if I run parsing from within the application where logs from different threads write into each other.

The issue is, when I do close the application, the parallel foreach loop still tries to process one more file before exiting. I want all tasks to stop immediately when I kill the application. Here is code excerpts:

My cancel is borrowed from: Capture console exit C#

Essentially the program performs some cleanup duties when it receives a cancel command such as CTRL+C or closing window with X button

The code I am trying to cancel is here:

static ConcurrentDictionary<int, Tuple<Tdx2KlarfParserProcInfo, string>> _currentProcessesConcurrentDict = new ConcurrentDictionary<int, Tuple<Tdx2KlarfParserProcInfo, string>>();
static bool exitSystem = false;

class Program
{
   
    private static bool _isFileLoadingDone;
    static ConcurrentDictionary<int, Tuple<Tdx2KlarfParserProcInfo, string>> _currentProcessesConcurrentDict = new ConcurrentDictionary<int, Tuple<Tdx2KlarfParserProcInfo, string>>();

    static void Main(string[] args)
    {
        try
        {
            if (args.Length == 0)
            {
                // Some boilerplate to react to close window event, CTRL-C, kill, etc
                ParseFilesUntilEmpty();
                while (!exitSystem)
                {
                    Thread.Sleep(500);
                }

            }

        }
    }

   
}

Which calls:

private static void LaunchFolderMode()
{
    //Some function launched from Task
    ParseFilesUntilEmpty();
}

And this calls:

private static void ParseFilesUntilEmpty()
{
    while (!_isFileLoadingDone)
    {
        ParseFiles();
    }
    
    ParseFiles();

}

Which calls:

private static void ParseFiles()
{
    filesToProcess = new HashSet<string>(){@"file1", "file2", "file3", "file4"} //I actuall get files from a db. this just for example
    //_fileStack = new ConcurrentStack<string>(filesToProcess);
    int parallelCount = 2
    Parallel.ForEach(filesToProcess, new ParallelOptions { MaxDegreeOfParallelism = parallelCount },
        tdxFile =>{
            ConfigureAndStartProcess(tdxFile);
        });
    
}

Which finally calls:

public static void ConfigureAndStartProcess(object fileName)
{
    string fileFullPath = fileName.ToString();
    Process proc = new Process();
    string fileFullPathArg1 = fileFullPath;
    string appName = @".\TDXXMLParser.exe";
    if (fileFullPathArg1.Contains(".gz"))
    {
        StartExe(appName, proc, fileFullPathArg1);  //I set up the arguments and launch the exes. And add the processes to _currentProcessesConcurrentDict
        proc.WaitForExit();
        _currentProcessesConcurrentDict.TryRemove(proc.Id, out Tuple<Tdx2KlarfParserProcInfo, string> procFileTypePair);
        proc.Dispose();
    }

}

The concurrent dictionary to monitor processes uses the following class in the tuple:

public class Tdx2KlarfParserProcInfo
{
    public int ProcId { get; set; }
    public List<long> MemoryAtIntervalList { get; set; } = new List<long>();
}

For the sake of how long these code excerpts are, I omitted the 'StartExe()' function. All it does is set up arguments and starts the process. Is there a better way parallel processing method I can use which will allow me to kill whatever files I am currently processing without immediately tryign to start a new process. Which the parallel.Foreach does?

I have tried killing it with Parallel State Stop method but it still tries to process one more file