I'm working on MIT 6.824 - lab1(part III) and get confused at a piece of scheduling program of a (mini) mapreduce:
var wg sync.WaitGroup
for i := 0; i < ntasks; i++ {
task_arg := DoTaskArgs{ JobName: jobName, File: mapFiles[i], Phase: phase, TaskNumber: i, NumOtherPhase: n_other }
//not so relevant
wg.Add(1)
go func() {
defer wg.Done()
reg_worker := <- registerChan
call(reg_worker, "Worker.DoTask", task_arg, nil)
go func() { registerChan <- reg_worker }()
//registerChan <- reg_worker
}()
}
wg.Wait()
This program
- deploy 100 tasks to 2 worker (with
call) from channelregisterChan - put worker back to channel when task finished
wg.Add(),wg.Done,wg.Wait()to sync
how schedule is called (https://github.com/WentaoZero/mini-mapreduce/blob/master/master.go#L84):
ch := make(chan string)
go mr.forwardRegistrations(ch)
schedule(mr.jobName, mr.files, mr.nReduce, phase, ch)
registerChan is not buffered.
I try removing goroutine at line go func() { registerChan <- reg_worker }() to make it: registerChan <- reg_worker
the program gets stuck after more than 50 tasks are done. I suppose it proves that the goroutine is working but I don't see why it gets stuck.
registerChan <- reg_worker has been written in a goroutine, why is it necessary to wrap it with another goroutine?
I don't think the rest of the system is relevant so I won't post it here. You may check https://github.com/WentaoZero/mini-mapreduce if needed. This scheduling program is picked from https://github.com/WentaoZero/mini-mapreduce/blob/master/schedule.go