valyala/fasthttp

fasthttp client. Read body in chunks and stop downloading on condition.

Open

#1,065 建立於 2021年8月5日

在 GitHub 查看
 (8 留言) (0 反應) (0 負責人)Go (21,741 star) (1,755 fork)batch import
help wanted

描述

For example, I want to stop downloading content if it has wrong mime type. How can I do so? The documentation shows only resp.Body() which gets whole response. Below is how I use standard http library:

resp, err := client.Do(req)
if err != nil {
	return nil, err
}
body := resp.Body
defer body.Close()

checkedMime := false
readBuffer := make([]byte, 1 << 12)
bodyData := make([]byte, 0, typicalSize)
for {
	n, err := body.Read(readBuffer)
	if err != nil && err != io.EOF {
		return nil, err
	}
	bodyData = append(bodyData, readBuffer[0:n]...)
	if !checkedMime && len(bodyData) > 512{			
		mimeType := http.DetectContentType(bodyData)
		if isArchive(mimeType){
			isArchive = true
			break
		}
		if mimeType  != "text/html"{
			isUselessContent = true
			break
		}
		checkedMime = true
	}
	if len(bodyData) > tooBigContentSize {
		tooBigContent = true
		break
	}
	if err == io.EOF {
		break
	}
}
// do something with bodyData, isArchive, isUselessContent, tooBigContent
...

貢獻者指南