🚀 go-pugleaf

blueman <NOSPAM@nospam.com> writes:
> blueman <NOSPAM@nospam.com> writes:
>
>> blueman <NOSPAM@nospam.com> writes:
>>
>>> Uday S Reddy <uDOTsDOTreddy@cs.bham.ac.uk> writes:
>>>> blueman wrote:
>>>
>>>> I played with some variations of it, and finally realized that the
>>>> problem must be the handling of the output.  In your test process, the
>>>> output is going to the testfile.  If I just use 'cat' as the command
>>>> instead of 'cat > testfile', then sure enough the problem
>>>> reappears. It starts hanging somewhere above 64K.
>>>
>>> Ahhhh... that's why I was asking for details on exactly what commands
>>> you used.
>>> That would explain our differences since cat'ing to a file (or running
>>> through spamassassin which is another test I did) generates only a
>>> handful of bytes of output.
>>
>> I can now reproduce your problem but it seems to happen *only* on
>> Windows. Interestingly, it happens using either bash.exe (Cygwin's
>> shell) or the Windows-like shell (cmdproxy.exe). So, seems to be
>> something different about how Emacs on Windows deals with processes
>>
>> Interestingly, when I use edebug to step through the function, it
>> doesn't hang as long as no single email is too large. If there is an
>> email larger than 64K, it hangs on the command:
>> (process-send-region process (nth 0 region) (nth 1 region)))
>>
>> So, it seems like there is some type of size/timing issue going on
>> here. Where a process cannot accept more than 64K either in one batch or
>> in batches too closely spaced together in time.
>>
>> And I can replicate it by running the following block from the
>> minibuffer on a buffer >64K in length (doesn't need to be mail) for
>> different values of NNN. The precise value where it hangs seems to
>> differ for different files but it always seems to be <64K (and it may be
>> due to the coding system).
>>
>> (progn (setq process (start-process "test" "tempbuff" shell-file-name
>> shell-command-switch "cat")) (process-send-region process (point-min) NNN))
>>
>>
>> NB.Just for a lark, I tested "cat" (without testfile) on a 50MB folder with
>> 2500 emails on Linux -- took about a minute on my 7 year old machine,
>> but worked flawlessly.
>>
>> So, clearly a bug in Emacs and nothing to do with VM or
>> vm-pipe-messages-to-command etc. Probably should be reported to the
>> Emacs team. Also, doesn't seem to be any way to fix it neatly unles you
>> would want to break up writes into smaller chunks (which is not that
>> bad in and of itself) and then wait for some arbitray amount of time
>> between writes (this seems just downright ugly & unnatural).
>>
>> BTW are you using ntEmacs or cygwin Emacs?
>>
>
> Interestingly, the problem is not simply related to either the size of
> the input or the size of the output. (The following is all only true for
> Windows):
>
> 1. "cat > tempfile"
>    - Doesn't crash regardless of input size
>
> 2. "cat"
>    - Seems to hang if input region >64K or so (sometimes a few K less,
>      sometimes a few hundred bytes more)
>
> 3. "cat file"
>    - Doesn't seem to work regardless of size of input region or file
>      (but works fine on Linux

I have file a bug report with the windows emacs team:
  http://debbugs.gnu.org/cgi/bugreport.cgi?bug=5359

Hopefully, this ends the long thread and in the future I will use
launchpad whenever possible!!!!

Unless I hear otherwise, I will assume this is a Windows emacs bug and
that there is nothing to fix within vm itself beyond the edits we have
discussed.

To avoid confusion, here is the cumulative bugfix and separator feature
enhancement against the 8.1.0-beta
--------------------------------------------------------------------------------

--- /usr/share/emacs/site-lisp/vm/vm-save.el	2009-12-19 14:26:10.000000000 -0500
+++ vm-save.el.new	2010-01-11 15:58:22.847588610 -0500
@@ -638,13 +638,12 @@
 	   current-prefix-arg)))
   (vm-pipe-message-to-command command prefix-arg t))
 
-(defun vm-pipe-command-exit-handler (process discard-output 
+(defun vm-pipe-command-exit-handler (process command discard-output 
 					     &optional exit-handler)
   "Switch to output buffer of PROCESS if DISCARD-OUTPUT non-nil.
-If non-nil call EXIT-HANDLER with the two arguments COMMAND and OUTPUT-BUFFER." 
+If non-nil call EXIT-HANDLER with the two arguments COMMAND and the buffer asociated with PROCESS."
   (let ((exit-code (process-exit-status process))
-	(buffer (process-buffer process))
-	(command (process-command process)))
+	(buffer (process-buffer process)))
   (if (not (zerop exit-code))
       (message "Command '%s' exit code is %d." command exit-code))
   (vm-display nil nil '(vm-pipe-message-to-command)
@@ -653,11 +652,17 @@
   (if exit-handler
       (funcall exit-handler command buffer))))
 
-(defvar vm-pipe-messages-to-command-start ""
-  "*Inserted by `vm-pipe-messages-to-command' before a message.")
-
-(defvar vm-pipe-messages-to-command-end "\n"
-  "*Inserted by `vm-pipe-messages-to-command' after a message.")
+(defvar vm-pipe-messages-to-command-start t
+  "*The string to be used as the leading message separator by
+`vm-pipe-messages-to-command' at the beginning of each message.
+If set to 't', then use the leading message separator stored in the VM
+folder. If nil then skip insertion.")
+
+(defvar vm-pipe-messages-to-command-end t
+  "*The string to be used as the trailing message separator by
+`vm-pipe-messages-to-command' at the end of each message.
+If set to 't', then use the trailing message separator stored in the VM
+folder. If nil then skip insertion")
 
 ;;;###autoload
 (defun vm-pipe-messages-to-command (command &optional prefix-arg 
@@ -669,8 +674,12 @@
 is much faster than calling the command on each message.  This is
 more like saving to a pipe.
 
-Before a message it will insert `vm-pipe-messages-to-command-start'
-and after a message `vm-pipe-messages-to-command-end'.
+Note that the interpretation of \\[universal-argument]'s is the same as for
+`vm-pipe-message-to-command'
+
+Before and after piping each message, it will add separators based on 
+the value of the variables `vm-pipe-messages-to-command-start'
+and `vm-pipe-messages-to-command-end', respectively. 
 
 Output, if any, is displayed unless DISCARD-OUTPUT is t.
 
@@ -712,24 +721,34 @@
 		(vm-pipe-command-exit-handler 
 		 process ,command ,discard-output 
 		 (if (and ,no-wait (functionp ,no-wait))
-		     no-wait)))
+		     ,no-wait)))
 	  (message "Command '%s' changed state to %s."
 		   ,command status))))
     (while mlist
       (setq m (vm-real-message-of (car mlist)))
       (set-buffer (vm-buffer-of m))
-      (process-send-string process vm-pipe-messages-to-command-start)
       (save-restriction
 	(widen)
+	(cond
+	 ((null vm-pipe-messages-to-command-start)) ; No start separator
+	 ((equal vm-pipe-messages-to-command-start t)
+	  (process-send-region process  ; Content before header
+				 (vm-start-of m) (vm-headers-of m)))
+	 (t (process-send-string process vm-pipe-messages-to-command-start)))
 	(let ((region (vm-pipe-message-part m prefix-arg)))
-	  (process-send-region process (nth 0 region) (nth 1 region))))
-      (process-send-string process vm-pipe-messages-to-command-end)
-      (setq mlist (cdr mlist)))
+	  (process-send-region process (nth 0 region) (nth 1 region)))
+	(cond
+	 ((null vm-pipe-messages-to-command-end)) ; No end separator
+	 ((equal vm-pipe-messages-to-command-end t)
+	  (process-send-region process  ; Content after end of body
+				 (vm-text-end-of m) (vm-end-of m)))
+	 ( t (process-send-string process vm-pipe-messages-to-command-end)))
+      (setq mlist (cdr mlist))))
 
     (process-send-eof process)
 
     (when (not no-wait) 
-      (while (and (eq 'run (process-status process)))
+      (while (eq 'run (process-status process))
 	(accept-process-output process)
 	(sit-for 0))
       (vm-pipe-command-exit-handler process command discard-output))