Have an interesting case I want to share and save for future. One small problem in this system: https://github.com/trianglesis/BMC_TPL_IDE is showing when some process is running and was not hanged up. Sometimes we need to run a huge amount of checks and tests and would be great to know if something hangs before killing it mannually. I used different scenarios and mostly a module pregressbar2, and this is what I finally can compose:

1. Make worker which will execute job sorf of randomly:

   1. import time
      import random
      
      print("START SOMETHING")
      
      for i in range(5):
          sec = random.randint(1, 3)
          print("Sleeping for: %d" % sec)
          time.sleep(sec)
          print("Making job, line: %d" % i)
      
      print("END SOMETHING")

2. Make process executor:

   1. import sys
      import subprocess
      import progressbar
      
      # Draw spinner:
      bar = progressbar.ProgressBar(max_value=progressbar.UnknownLength)
      
      # Execute some job with multiple lines on stdout:
      p = subprocess.Popen("C:\Python34\python.exe worker.py", shell=True, stdout=subprocess.PIPE,
                                                                           stderr=subprocess.PIPE)
      # Lines will be collected in list:
      result = []
      
      # Until I get last line and the end of string:
      while p.stdout is not None:
      
          # Update spinner on one step:
          # It will update only when any line was printed to stdout!
          bar.update()
          # Read each line:
      
          line = p.stdout.readline()
          # Add line in list and remove carriage return
      
          result.append(line.decode('UTF-8').rstrip('\r'))
      
          # When no lines appears:
          if not line:
              print("\n")
              p.stdout.flush()
              break
      
      # Show finish message, it also useful because bar cannot start new line on console, why?
      print("Finish:")
      # Results as string:
      print(''.join(result))

       

Voilla! But there is still a small problem - this bar will update it's state only when process post something in stdout. Between posts it will stay calm for random amount of time (as designed in worker). If I'll update while or somehow try to make this loop faster - it becomes a deadlock.    Next example is from: https://github.com/trianglesis/BMC_TPL_IDE Function: tests_executor. In this example - bar works better, because It have a list of tests, so I can enumerate its len and config bar to show how much progress were already done. But yes, it will update only each one itetarion, and till test run, for 600+ sec - it wouldn't move.

    def tests_executor(self, tests_list):

        tests_len = len(tests_list)

        if progressbar:
            progressbar.streams.flush()
            progressbar.streams.wrap_stderr()
            bar = progressbar.ProgressBar(max_value=tests_len)
        else:
            log.debug("Module progressbar2 is not installed, will show progress in usual manner.")
            pass

        log.info("-==== START RELATED TESTS EXECUTION ====-")
        log.debug("Tests related to: "+str(tests_list[0]['pattern']))
        log.debug("All tests len: "+str(tests_len))

        for i, test in enumerate(tests_list):

            log.info("Start test:" + str(test['rem_test_path']))

            if progressbar:
                bar(range(tests_len))
                bar.update(i)
            else:
                pass

            log.info("%d test of "+str(tests_len), i+1)  # Just print

            pre_cmd = ". ~/.bash_profile;"
            wd_cmd = "cd "+test['rem_test_wd']+";"
            cmd_test = "/usr/tideway/bin/python -u "+test['rem_test_path']+" --verbose"

            cmd = pre_cmd + wd_cmd + cmd_test
            log.debug("Run: "+str(cmd))

            try:
                _, stdout, stderr = self.ssh_cons.exec_command(cmd)

                if stdout:
                    output = stdout.readlines()
                    raw_out = "".join(output)
                    log.info("-==== DETAILED LOG ====-")
                    log.info("\n"+raw_out)

                if stderr:
                    output = stderr.readlines()
                    raw_out = "".join(output)
                    log.info("-==== UNITTEST LOG ====-")
                    log.info("\n\n"+raw_out)
            except:
                log.error("Test execution command cannot run: "+str(cmd))

        if progressbar:
            bar.finish()
        else:
            pass

        log.info("-==== END OF RELATED TESTS EXECUTION ====-")

  So the problem solved, partially.  Still wondering - how to make progressbar or spinner update constantly till process not die?