微服务SpringCloud项目(四):整合MinIo实现文

小知识,大挑战!本文正在参与「程序员必备小知识」创作活动

本文已参与 「掘力星计划」 ,赢取创作大礼包,挑战创作激励金。

📖前言

1
复制代码心态好了,就没那么累了。心情好了,所见皆是明媚风景。

“一时解决不了的问题,那就利用这个契机,看清自己的局限性,对自己进行一场拨乱反正。”正如老话所说,一念放下,万般自在。如果你正被烦心事扰乱心神,不妨学会断舍离。断掉胡思乱想,社区垃圾情绪,离开负面能量。心态好了,就没那么累了。心情好了,所见皆是明媚风景。

🚓进入正题

不多说了:用的mybatis-plus具体的实现类什么的就不写了,毕竟复制粘贴谁都会希望你不是复制粘贴一把梭呵呵,进入正题吧

结构如下

1632966052.png

1. 创建子模块单独存放配置读取和工具类(为了演示方便我就放在一个项目里了别介意)

引入如下依赖:

1
2
3
4
5
6
xml复制代码<!-- https://mvnrepository.com/artifact/io.minio/minio -->
<dependency>
<groupId>io.minio</groupId>
<artifactId>minio</artifactId>
<version>${minio.version}</version>
</dependency>

yml增加配置如下,其他配置自行搞定

1
2
3
4
5
6
7
8
yaml复制代码# Minio配置
minio:
server:
url: mini访问地址
accessKey: 密钥
secretKey: 密钥
originFileBucKetValue: dream-cloud-auth # 存储桶需要验证
allowOriginFileBucKetValue: dream-cloud-allow # 存储桶放行上传

2. MinIoProperties.java – 用于 Minio 配置信息获取

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
java复制代码package com.cyj.dream.file.config;

import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.cloud.context.config.annotation.RefreshScope;
import org.springframework.context.annotation.Configuration;

/**
* @Description: Minio配置信息获取
* @BelongsProject: DreamChardonnay
* @BelongsPackage: com.cyj.dream.file.config
* @Author: ChenYongJia
* @CreateTime: 2021-09-27
* @Email: chen87647213@163.com
* @Version: 1.0
*/
@Data
@RefreshScope
@Configuration
@ConfigurationProperties("minio.server")
public class MinIoProperties {

/**
* minio地址--url+端口号
*/
private String url;

/**
* 账号
*/
private String accessKey;

/**
* 密码
*/
private String secretKey;

/**
* 分区配置
*/
private String chunkBucKetValue;

/**
* 桶名配置(限权的)
*/
private String originFileBucKetValue;

/**
* 桶名配置(放行的)
*/
private String allowOriginFileBucKetValue;

}

3. MinIoUtils.java – 用于操作 MinIo 工具类

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
java复制代码package com.cyj.dream.minio.util;

import cn.hutool.core.date.DateUtil;
import cn.hutool.core.lang.UUID;
import cn.hutool.core.util.StrUtil;
import com.cyj.dream.minio.config.MinIoProperties;
import com.google.common.io.ByteStreams;
import io.minio.*;
import io.minio.errors.MinioException;
import io.minio.http.Method;
import io.minio.messages.Item;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import javax.annotation.PostConstruct;
import javax.annotation.Resource;
import java.io.*;
import java.util.*;
import java.util.stream.Collectors;

/**
* @Description: MinIo工具类
* @BelongsProject: DreamChardonnay
* @BelongsPackage: com.cyj.dream.minio.util
* @Author: ChenYongJia
* @CreateTime: 2021-09-26
* @Email: chen87647213@163.com
* @Version: 1.0
*/
@Slf4j
@Component
public class MinIoUtils {

@Autowired
private MinIoProperties minIoProperties;

private static String url;

private static String accessKey;

private static String secretKey;

public static String chunkBucKet;

public static String originFileBucKet;

public static String allowOriginFileBucKet;

private static MinioClient minioClient;

/**
* 排序
*/
public final static boolean SORT = true;

/**
* 不排序
*/
public final static boolean NOT_SORT = false;

/**
* 默认过期时间(分钟)
*/
private final static Integer DEFAULT_EXPIRY = 60;

/**
* 初始化MinIo对象
*/
@PostConstruct
public void init() {
url = minIoProperties.getUrl();
accessKey = minIoProperties.getAccessKey();
secretKey = minIoProperties.getSecretKey();
chunkBucKet = minIoProperties.getChunkBucKetValue();
originFileBucKet = minIoProperties.getOriginFileBucKetValue();
allowOriginFileBucKet = minIoProperties.getAllowOriginFileBucKetValue();
}

public static void afterPropertiesSet() throws Exception {
log.info("url ====>{}", url);
minioClient = MinioClient.builder().endpoint(url).credentials(accessKey, secretKey).build();
//方便管理分片文件,则单独创建一个分片文件的存储桶
if (!StrUtil.isEmpty(chunkBucKet) && !isBucketExist(chunkBucKet)) {
createBucket(chunkBucKet);
}
if (!StrUtil.isEmpty(originFileBucKet) && !isBucketExist(originFileBucKet)) {
createBucket(originFileBucKet);
}
if (!StrUtil.isEmpty(allowOriginFileBucKet) && !isBucketExist(allowOriginFileBucKet)) {
createBucket(allowOriginFileBucKet);
}
}

/**
* 获得实例
* @author ChenYongJia
* @date 2021-9-26 09:33:18
* @return io.minio.MinioClient
*/
public static MinioClient getInstance() throws Exception {

if (minioClient == null) {
synchronized (MinIoUtils.class) {
if (minioClient == null) {
afterPropertiesSet();
}
}
}
return minioClient;
}

/**
* 存储桶是否存在
*
* @param bucketName 存储桶名称
* @return true/false
*/
public static boolean isBucketExist(String bucketName) throws Exception {
return getInstance().bucketExists(BucketExistsArgs.builder().bucket(bucketName).build());
}

/**
* 上传对象到minio,目标位置和存储名称存在重复时会覆盖
*
* @param bucketName 目标桶
* @param filePath 目标位置和存储名称,如 tempdir/123.txt
* @param file
*/
public static void putObject(String bucketName, String filePath, File file) throws Exception {
getInstance().putObject(
PutObjectArgs.builder()
.bucket(bucketName)
.object(removeSlash(filePath))
.stream(new FileInputStream(file), file.length(), -1)
.build()
);
}

/**
* 桶内文件复制
*
* @param bucketName 目标桶
* @param sourceFilePath 目标位置
* @param targetFilePath 复制到
*/
public static boolean copyObject(String bucketName, String sourceFilePath, String targetFilePath) throws Exception {

try {
getInstance().copyObject(CopyObjectArgs.builder()
.bucket(bucketName)
.object(targetFilePath)
.source(CopySource
.builder()
.bucket(bucketName)
.object(sourceFilePath)
.build()
)
.build());
return true;
} catch (MinioException e) {
System.out.println("Error occurred: " + e);
return false;
}

}

/**
* 上传对象到minio,目标位置和存储名称存在重复时会覆盖
*
* @param bucketName 目标桶
* @param filePath 目标位置和存储名称,如 tempdir/123.txt
* @param file
*/
public static void putObject(String bucketName, String filePath, InputStream file) throws Exception {
getInstance().putObject(
PutObjectArgs.builder()
.bucket(bucketName)
.object(removeSlash(filePath))
.stream(file, file.available(), -1)
.build()
);
}

/**
* 删除削减
*
* @param str 入参
* @author ChenYongJia
* @date 9:33 2021/9/26
* @return * @return java.lang.String
*/
private static String removeSlash(String str) {
if (str.substring(0, 1).equals("/")) {
return str.substring(1);
}
return str;
}

/**
* 从minio下载指定路径对象,目标位置和存储名称存在重复时会覆盖
*
* @param bucketName 目标桶
* @param filePath 目标位置和存储名称,如 tempdir/123.txt
*/
public static byte[] getObject(String bucketName, String filePath) throws Exception {

InputStream inputStream = null;
try {
inputStream = getInstance().getObject(
GetObjectArgs.builder()
.bucket(bucketName)
.object(removeSlash(filePath))
.build()
);
} catch (MinioException e) {
System.out.println("Error occurred: " + e);
return null;
}
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
{
ByteStreams.copy(inputStream, outputStream);
byte[] buffer = outputStream.toByteArray();
return buffer;
}
}

/**
* 获取文件状态信息
*
* @param bucketName 目标桶
* @param filePath 目标位置和存储名称,如 tempdir/123.txt
*/
public static StatObjectResponse statObject(String bucketName, String filePath) throws Exception {

try {
return getInstance().statObject(
StatObjectArgs.builder()
.bucket(bucketName)
.object(removeSlash(filePath))
.build()
);
} catch (MinioException e) {
System.out.println("Error occurred: " + e);
return null;
}
}

/**
* 移除文件
*
* @param bucketName 目标桶
* @param filePath 目标位置和存储名称,如 tempdir/123.txt
*/
public static void removeObject(String bucketName, String filePath) throws Exception {
getInstance().removeObject(
RemoveObjectArgs.builder()
.bucket(bucketName)
.object(removeSlash(filePath))
.build()
);
}

/**
* 创建存储桶
*
* @param bucketName 存储桶名称
* @return true/false
*/
public static boolean createBucket(String bucketName) throws Exception {
getInstance().makeBucket(MakeBucketArgs.builder().bucket(bucketName).build());
return true;
}

/**
* 获取访问对象的外链地址
*
* @param objectName 对象名称
* @param expiry 过期时间(分钟) 最大为7天 超过7天则默认最大值
* @return viewUrl
*/
public static String getOriginalObjectUrl(String objectName, Integer expiry) throws Exception {
return getObjectUrl(originFileBucKet, objectName, expiry);
}

/**
* 获取访问对象的外链地址
*
* @param bucketName 存储桶名称
* @param objectName 对象名称
* @param expiry 过期时间(分钟) 最大为7天 超过7天则默认最大值
* @return viewUrl
*/
public static String getObjectUrl(String bucketName, String objectName, Integer expiry) throws Exception {
expiry = expiryHandle(expiry);
return getInstance().getPresignedObjectUrl(
GetPresignedObjectUrlArgs.builder()
.method(Method.GET)
.bucket(bucketName)
.object(removeSlash(objectName))
.expiry(expiry)
.build()
);
}

/**
* 创建上传文件对象的外链
*
* @param bucketName 存储桶名称
* @param objectName 欲上传文件对象的名称
* @param expiry 过期时间(分钟) 最大为7天 超过7天则默认最大值
* @return uploadUrl
*/
public static String createUploadUrl(String bucketName, String objectName, Integer expiry) throws Exception {
expiry = expiryHandle(expiry);
return getInstance().getPresignedObjectUrl(
GetPresignedObjectUrlArgs.builder()
.method(Method.PUT)
.bucket(bucketName)
.object(removeSlash(objectName))
.expiry(expiry)
.build()
);
}

/**
* 创建上传文件对象的外链
*
* @param bucketName 存储桶名称
* @param objectName 欲上传文件对象的名称
* @return uploadUrl
*/
public static String createUploadUrl(String bucketName, String objectName) throws Exception {
return createUploadUrl(bucketName, objectName, DEFAULT_EXPIRY);
}

/**
* 批量创建分片上传外链
*
* @param bucketName 存储桶名称
* @param objectMD5 欲上传分片文件主文件的MD5
* @param chunkCount 分片数量
* @return uploadChunkUrls
*/
public static List<String> createUploadChunkUrlList(String bucketName, String objectMD5, Integer chunkCount) throws Exception {
if (null == bucketName) {
bucketName = chunkBucKet;
}
if (null == objectMD5) {
return null;
}
objectMD5 += "/";
if (null == chunkCount || 0 == chunkCount) {
return null;
}
List<String> urlList = new ArrayList<>(chunkCount);
for (int i = 1; i <= chunkCount; i++) {
String objectName = objectMD5 + i + ".chunk";
urlList.add(createUploadUrl(bucketName, objectName, DEFAULT_EXPIRY));
}
return urlList;
}

/**
* 创建指定序号的分片文件上传外链
*
* @param bucketName 存储桶名称
* @param objectMD5 欲上传分片文件主文件的MD5
* @param partNumber 分片序号
* @return uploadChunkUrl
*/
public static String createUploadChunkUrl(String bucketName, String objectMD5, Integer partNumber) throws Exception {
if (null == bucketName) {
bucketName = chunkBucKet;
}
if (null == objectMD5) {
return null;
}
objectMD5 += "/" + partNumber + ".chunk";
return createUploadUrl(bucketName, objectMD5, DEFAULT_EXPIRY);
}

/**
* 获取对象文件名称列表
*
* @param bucketName 存储桶名称
* @param prefix 对象名称前缀
* @param sort 是否排序(升序)
* @return objectNames
*/
public static List<String> listObjectNames(String bucketName, String prefix, Boolean sort) throws Exception {
ListObjectsArgs listObjectsArgs;
if (null == prefix) {
listObjectsArgs = ListObjectsArgs.builder()
.bucket(bucketName)
.recursive(true)
.build();
} else {
listObjectsArgs = ListObjectsArgs.builder()
.bucket(bucketName)
.prefix(prefix)
.recursive(true)
.build();
}
Iterable<Result<Item>> chunks = getInstance().listObjects(listObjectsArgs);
List<String> chunkPaths = new ArrayList<>();
for (Result<Item> item : chunks) {
chunkPaths.add(item.get().objectName());
}
if (sort) {
return chunkPaths.stream().distinct().collect(Collectors.toList());
}
return chunkPaths;
}

/**
* 获取对象文件名称列表
*
* @param bucketName 存储桶名称
* @param prefix 对象名称前缀
* @return objectNames
*/
public static List<String> listObjectNames(String bucketName, String prefix) throws Exception {
return listObjectNames(bucketName, prefix, NOT_SORT);
}

/**
* 获取分片文件名称列表
*
* @param bucketName 存储桶名称
* @param ObjectMd5 对象Md5
* @return objectChunkNames
*/
public static List<String> listChunkObjectNames(String bucketName, String ObjectMd5) throws Exception {
if (null == bucketName) {
bucketName = chunkBucKet;
}
if (null == ObjectMd5) {
return null;
}
return listObjectNames(bucketName, ObjectMd5, SORT);
}

/**
* 获取分片名称地址HashMap key=分片序号 value=分片文件地址
*
* @param bucketName 存储桶名称
* @param ObjectMd5 对象Md5
* @return objectChunkNameMap
*/
public static Map<Integer, String> mapChunkObjectNames(String bucketName, String ObjectMd5) throws Exception {
if (null == bucketName) {
bucketName = chunkBucKet;
}
if (null == ObjectMd5) {
return null;
}
List<String> chunkPaths = listObjectNames(bucketName, ObjectMd5);
if (null == chunkPaths || chunkPaths.size() == 0) {
return null;
}
Map<Integer, String> chunkMap = new HashMap<>(chunkPaths.size());
for (String chunkName : chunkPaths) {
Integer partNumber = Integer.parseInt(chunkName.substring(chunkName.indexOf("/") + 1, chunkName.lastIndexOf(".")));
chunkMap.put(partNumber, chunkName);
}
return chunkMap;
}

/**
* 合并分片文件成对象文件
*
* @param chunkBucKetName 分片文件所在存储桶名称
* @param composeBucketName 合并后的对象文件存储的存储桶名称
* @param chunkNames 分片文件名称集合
* @param objectName 合并后的对象文件名称
* @return true/false
*/
public static boolean composeObject(String chunkBucKetName, String composeBucketName, List<String> chunkNames, String objectName) throws Exception {
if (null == chunkBucKetName) {
chunkBucKetName = chunkBucKet;
}
List<ComposeSource> sourceObjectList = new ArrayList<>(chunkNames.size());
for (String chunk : chunkNames) {
sourceObjectList.add(
ComposeSource.builder()
.bucket(chunkBucKetName)
.object(chunk)
.build()
);
}
getInstance().composeObject(
ComposeObjectArgs.builder()
.bucket(composeBucketName)
.object(removeSlash(objectName))
.sources(sourceObjectList)
.build()
);
return true;
}

/**
* 合并分片文件成对象文件
*
* @param chunkNames 分片文件名称集合
* @param objectName 合并后的对象文件名称
* @return true/false
*/
public static boolean composeObject(List<String> chunkNames, String objectName) throws Exception {
return composeObject(chunkBucKet, originFileBucKet, chunkNames, objectName);
}

/**
* 直接上传
*
* @param fileName 文件名
* @param inputStream 文件流
* @return
* @throws Exception
*/
public static String upload(String fileName, InputStream inputStream, String fileBucketName) throws Exception {
String suffix = fileName.substring(fileName.lastIndexOf(".") + 1);
return upload(inputStream, suffix, fileBucketName);
}

/**
* 直接上传
*
* @param inputStream 文件流
* @param suffix 文件后缀
* @return
* @throws Exception
*/
public static String upload(InputStream inputStream, String suffix, String fileBucketName) throws Exception {
String uuid = UUID.randomUUID().toString();
String savePath = getSavePath(uuid + "." + suffix);
putObject(fileBucketName, savePath, inputStream);
return savePath;
}

private static String getSavePath(String fileName) {

String dayStr = DateUtil.now();
String days = dayStr.substring(0, dayStr.lastIndexOf(" "));
String[] dayArr = days.split("-");

String path = dayArr[0] + "/" + dayArr[1] + "/" + dayArr[2] + "/" + fileName;

return path;
}

/**
* 将分钟数转换为秒数
*
* @param expiry 过期时间(分钟数)
* @return expiry
*/
private static int expiryHandle(Integer expiry) {
expiry = expiry * 60;
if (expiry > 604800) {
return 604800;
}
return expiry;
}

}

4. 文件中心常量

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
java复制代码package com.cyj.dream.file.contacts;

/**
* @Description: 文件中心常量
* @BelongsProject: DreamChardonnay
* @BelongsPackage: com.cyj.dream.file.contacts
* @Author: ChenYongJia
* @CreateTime: 2021-09-26
* @Email: chen87647213@163.com
* @Version: 1.0
*/
public class FileConstant {

/**
* 包目录
*/
public static final String SCAN_BASE_PACKAGES_URL = "com.cyj.dream.file";

/**
* mapper文件目录
*/
public static final String MAPPER_URL = "com.cyj.dream.file.mapper";

/**
* 上传
*/
public final static String UPLOAD_TYPE_AUTH="auth";

/**
*
*/
public final static String UPLOAD_TYPE_ALLOW="allow";

}

5. 创建操作数据库实体

FileUploadPieceRecord – 文件分片断点续传记录表

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
java复制代码package com.cyj.dream.file.model;

import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import com.cyj.dream.core.constant.TreeEntity;
import io.swagger.annotations.ApiModel;
import io.swagger.annotations.ApiModelProperty;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.ToString;

import javax.persistence.*;

/**
* @Description: 文件分片断点续传记录表
* @BelongsProject: DreamChardonnay
* @BelongsPackage: com.cyj.dream.file.model
* @Author: ChenYongJia
* @CreateTime: 2021-09-13
* @Email: chen87647213@163.com
* @Version: 1.0
*/
@Data
@Entity
@ToString
@Table(name = "file_upload_piece_record")
@TableName("file_upload_piece_record")
@ApiModel(value = "FileUploadPieceRecord", description = "file_upload_piece_record 分片上传记录表")
public class FileUploadPieceRecord extends TreeEntity {

@Id
@TableId(value = "piece_id", type = IdType.AUTO)
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(nullable = false, columnDefinition = "bigint(20) unsigned COMMENT '分片上传记录表主键--自增'")
@ApiModelProperty(value = "文件路径id", example = "1")
private Long pieceId;

/**
* varchar(128) 分片存储桶名称
*/
@ApiModelProperty(value = "分片存储桶名称")
@Column(columnDefinition = "varchar(128) COMMENT '文件路径'")
private String chunkBucketName;

/**
* varchar(100) 源文件存储桶名称
*/
@ApiModelProperty(value = "源文件存储桶名称")
@Column(columnDefinition = "varchar(100) COMMENT '源文件存储桶名称'")
private String fileBucketName;

/**
* 分片数量
*/
@ApiModelProperty(value = "分片数量")
@Column(columnDefinition = "bigint(20) COMMENT '分片数量'")
private Long chunkCount;

/**
* varchar(255) 上传文件的md5
*/
@ApiModelProperty(value = "上传文件的md5")
@Column(columnDefinition = "varchar(255) COMMENT '上传文件的md5'")
private String fileMd5;

/**
* varchar(100) 上传文件/合并文件的格式
*/
@ApiModelProperty(value = "上传文件/合并文件的格式")
@Column(columnDefinition = "varchar(64) COMMENT '上传文件/合并文件的格式'")
private String fileSuffix;

/**
* 文件名称
*/
@ApiModelProperty(value = "文件名称")
@Column(columnDefinition = "varchar(128) COMMENT '文件名称'")
private String fileName;

/**
* varchar(255) 文件大小(b)
*/
@ApiModelProperty(value = "文件大小(b)")
@Column(columnDefinition = "varchar(255) COMMENT '文件大小(b)'")
private String fileSize;

/**
* 文件地址 varchar(500)
*/
@ApiModelProperty(value = "文件地址")
@Column(columnDefinition = "varchar(500) COMMENT '文件地址'")
private String filePath;

/**
* 上传状态 0.上传完成 1.已上传部分 int(11)
*/
@ApiModelProperty(value = "上传状态")
@Column(columnDefinition = "int(2) COMMENT '上传状态 0.上传完成 1.已上传部分 int(2)'")
private Integer uploadStatus;

/**
* 分片序号
*/
@Transient
private Integer partNumber;

/**
* 上传地址
*/
@Transient
private String uploadUrl;

}

FileUploadRecord – 文件上传记录

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
java复制代码package com.cyj.dream.file.model;

import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import com.cyj.dream.core.constant.BaseEntity;
import io.swagger.annotations.ApiModel;
import io.swagger.annotations.ApiModelProperty;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.ToString;

import javax.persistence.*;

/**
* @Description: 文件上传记录
* @BelongsProject: DreamChardonnay
* @BelongsPackage: com.cyj.dream.file.model
* @Author: ChenYongJia
* @CreateTime: 2021-09-13
* @Email: chen87647213@163.com
* @Version: 1.0
*/
@Data
@Entity
@ToString
@Table(name = "file_upload_record")
@TableName("file_upload_record")
@ApiModel(value = "FileUploadRecord", description = "file_upload_record 文件上传记录")
public class FileUploadRecord extends BaseEntity {

@ApiModelProperty(value = "文件路径id", example = "1")
@Id
@TableId(value = "file_id", type = IdType.AUTO)
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(nullable = false, columnDefinition = "bigint(20) unsigned COMMENT '文件上传记录主键--自增'")
private Long fileId;

@ApiModelProperty(value = "文件路径 varchar(500)")
@Column(columnDefinition = "varchar(500) COMMENT '文件路径'")
private String filePath;

@ApiModelProperty(value = "文件名称 varchar(255)")
@Column(columnDefinition = "varchar(255) COMMENT '文件名称'")
private String fileName;

@ApiModelProperty(value = "上传文件/合并文件的格式 varchar(64)")
@Column(columnDefinition = "varchar(64) COMMENT '上传文件/合并文件的格式'")
private String fileSuffix;

@ApiModelProperty(value = "源文件存储桶名称 varchar(128)")
@Column(columnDefinition = "varchar(128) COMMENT '源文件存储桶名称'")
private String fileBucketName;

@ApiModelProperty(value = "类型(allow 是放行,auth 是需要鉴权的 ) varchar(128)")
@Column(columnDefinition = "varchar(128) COMMENT '类型(allow 是放行,auth 是需要鉴权的 )'")
private String fileType;

}

6. FileManagementServiceImpl – 列一下文件管理实现类

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
java复制代码package com.cyj.dream.file.service.impl;

import cn.hutool.core.util.ObjectUtil;
import cn.hutool.core.util.StrUtil;
import com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;
import com.cyj.dream.core.constant.pagemodel.ResponseUtil;
import com.cyj.dream.core.util.date.DateUtils;
import com.cyj.dream.file.contacts.FileConstant;
import com.cyj.dream.file.mapper.FileUploadPieceRecordMapper;
import com.cyj.dream.file.mapper.FileUploadRecordMapper;
import com.cyj.dream.file.model.FileUploadPieceRecord;
import com.cyj.dream.file.model.FileUploadRecord;
import com.cyj.dream.file.service.FileManagementService;
import com.cyj.dream.file.util.MinIoUtils;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Service;
import org.springframework.web.multipart.MultipartFile;

import javax.annotation.Resource;
import java.time.LocalDateTime;
import java.util.*;

/**
* @Description: 文件管理实现类
* @BelongsProject: DreamChardonnay
* @BelongsPackage: com.cyj.dream.file.service.impl
* @Author: ChenYongJia
* @CreateTime: 2021-09-13
* @Email: chen87647213@163.com
* @Version: 1.0
*/
@Slf4j
@Service
public class FileManagementServiceImpl implements FileManagementService {

/**
* 上传成功
*/
private final Integer UPLOAD_SUCCESS = 1;

/**
* 部分上传
*/
private final Integer UPLOAD_PART = 0;

@Resource
private FileUploadRecordMapper fileUploadRecordMapper;

@Resource
private FileUploadPieceRecordMapper fileUploadPieceRecordMapper;

@Override
public Object initChunkUpload(String md5, String chunkCount, String fileSize) {
try {
FileUploadPieceRecord uploadDto = new FileUploadPieceRecord();
uploadDto.setFileSize(fileSize);
uploadDto.setFileMd5(md5);
uploadDto.setChunkCount(Long.valueOf(chunkCount));
QueryWrapper<FileUploadPieceRecord> queryWrapper = new QueryWrapper<>();
queryWrapper.lambda().eq(FileUploadPieceRecord::getFileMd5, md5);
FileUploadPieceRecord mysqlFileData = fileUploadPieceRecordMapper.selectOne(queryWrapper);
if (ObjectUtil.isNotEmpty(mysqlFileData)) {
//秒传
if (UPLOAD_SUCCESS.equals(mysqlFileData.getUploadStatus())) {
return mysqlFileData;
}
//续传
//获取到该文件已上传分片
Map<Integer, String> okChunkMap = MinIoUtils.mapChunkObjectNames(MinIoUtils.chunkBucKet, uploadDto.getFileMd5());
List<FileUploadPieceRecord> chunkUploadUrls = new ArrayList<>();
if (ObjectUtil.isNotEmpty(okChunkMap) && okChunkMap.size() > 0) {
for (int i = 1; i <= uploadDto.getChunkCount(); i++) {
//判断当前分片是否已经上传过了
if (!okChunkMap.containsKey(i)) {
//生成分片上传url
FileUploadPieceRecord url = new FileUploadPieceRecord();
url.setPartNumber(i);
url.setUploadUrl(MinIoUtils.createUploadChunkUrl(MinIoUtils.chunkBucKet, uploadDto.getFileMd5(), i));
chunkUploadUrls.add(url);
}
}
if (chunkUploadUrls.size() == 0) {
return "所有分片已经上传完成,仅需要合并文件";
}
return chunkUploadUrls;
}
}
//初次上传和已有文件信息但未上传任何分片的情况下则直接生成所有上传url
List<String> uploadUrls = MinIoUtils.createUploadChunkUrlList(MinIoUtils.chunkBucKet, uploadDto.getFileMd5(),
Integer.valueOf(uploadDto.getChunkCount().toString()));
List<FileUploadPieceRecord> chunkUploadUrls = new ArrayList<>();
for (int i = 1; i <= uploadUrls.size(); i++) {
FileUploadPieceRecord url = new FileUploadPieceRecord();
url.setPartNumber(i);
url.setUploadUrl(uploadUrls.get(i - 1));
chunkUploadUrls.add(url);
}
//向数据库中记录该文件的上传信息
uploadDto.setUploadStatus(UPLOAD_PART);
if (ObjectUtil.isEmpty(mysqlFileData)) {
uploadDto.setChunkBucketName(MinIoUtils.chunkBucKet);
uploadDto.setFileBucketName(MinIoUtils.originFileBucKet);
uploadDto.setCreateTime(DateUtils.toLocalDateTime(new Date()));
uploadDto.setUpdateTime(DateUtils.toLocalDateTime(new Date()));
fileUploadPieceRecordMapper.insert(uploadDto);
}
return chunkUploadUrls;
} catch (Exception ex) {
log.error("发生异常,异常信息为:{}", ex);
return ResponseUtil.error("初始化文件失败");
}
}

@Override
public Object composeFile(String md5, String fileName) {
try {
//根据md5获取所有分片文件名称(minio的文件名称 = 文件path)
List<String> chunks = MinIoUtils.listObjectNames(MinIoUtils.chunkBucKet, md5);
//获取过来的分片文件名称是乱序的,所以重新组合排序一下
List<String> newChunks = new ArrayList<>(chunks.size());
for (int i = 1; i <= chunks.size(); i++) {
String newChunkName = md5 + "/" + i + ".chunk";
newChunks.add(newChunkName);
}
//自定义文件名称
String suffix = fileName.substring(fileName.lastIndexOf("."));
String filePath = md5 + "/" + fileName;
//合并文件
if (!MinIoUtils.composeObject(newChunks, filePath)) {
return ResponseUtil.error("合并文件失败");
}
String url = null;
try {
//获取文件访问外链(1小时过期)
url = MinIoUtils.getOriginalObjectUrl(filePath, 60);
} catch (Exception e) {
log.error("发生异常,异常信息为:{}", e);
return ResponseUtil.error("获取文件下载连接失败");
}
//获取数据库里记录的文件信息,修改数据并返回文件信息
QueryWrapper<FileUploadPieceRecord> queryWrapper = new QueryWrapper<>();
queryWrapper.lambda().eq(FileUploadPieceRecord::getFileMd5, md5);
FileUploadPieceRecord dbData = fileUploadPieceRecordMapper.selectOne(queryWrapper);
dbData.setUploadStatus(UPLOAD_SUCCESS);
dbData.setFileName(fileName);
dbData.setFilePath(url);
dbData.setFileSuffix(suffix);

dbData.setCreateTime(DateUtils.toLocalDateTime(new Date()));
dbData.setUpdateTime(DateUtils.toLocalDateTime(new Date()));
//更新数据库中的附件上传状态
fileUploadPieceRecordMapper.updateById(dbData);
return dbData;
} catch (Exception ex) {
log.error("发生异常,异常信息为:{}", ex);
return ResponseUtil.error("合并失败");
}
}

@Override
public Object authUpload(MultipartFile file, String fileName) {
return this.upload(file, fileName, FileConstant.UPLOAD_TYPE_AUTH, MinIoUtils.originFileBucKet);
}

@Override
public Object allowUpload(MultipartFile file, String fileName) {
return this.upload(file, fileName, FileConstant.UPLOAD_TYPE_ALLOW, MinIoUtils.allowOriginFileBucKet);
}

@Override
public Object getPicBase64(FileUploadRecord fileUploadRecord) {
Map<String, String> result = new HashMap<>(16);
try {
byte[] file = new byte[0];
if (!StrUtil.isEmpty(fileUploadRecord.getFilePath())) {
file = MinIoUtils.getObject(fileUploadRecord.getFileBucketName(), fileUploadRecord.getFilePath());
}
String encoded = Base64.getEncoder().encodeToString(file);
String type = "";
if (fileUploadRecord.getFilePath().toLowerCase().contains(".jpg")) {
type = "data:image/jpeg;base64,";
} else if (fileUploadRecord.getFilePath().toLowerCase().contains(".png")) {
type = "data:image/png;base64,";
} else if (fileUploadRecord.getFilePath().toLowerCase().contains(".gif")) {
type = "data:image/gif;base64,";
}
result.put("base64", type + encoded);
} catch (Exception ex) {
log.error("发生异常,异常信息为:{}", ex);
return ResponseUtil.error("获取异常");
}
return result;
}

/**
* 上传
*
* @param file
* @param fileName
* @param type
* @param fileBucketName
* @return
*/
private Object upload(MultipartFile file, String fileName, String type, String fileBucketName) {
try {
String suffix = fileName.substring(fileName.lastIndexOf(".") + 1);
String saveUrl = MinIoUtils.upload(file.getInputStream(), suffix, fileBucketName);
FileUploadRecord fileUploadRecord = new FileUploadRecord();
fileUploadRecord.setFileBucketName(fileBucketName);
fileUploadRecord.setFileName(fileName);
fileUploadRecord.setFileSuffix(suffix);
fileUploadRecord.setFilePath(saveUrl);
fileUploadRecord.setFileType(type);

fileUploadRecord.setCreateTime(DateUtils.toLocalDateTime(new Date()));
fileUploadRecord.setUpdateTime(DateUtils.toLocalDateTime(new Date()));
fileUploadRecordMapper.insert(fileUploadRecord);
return fileUploadRecord;
} catch (Exception ex) {
log.error("发生异常,异常信息为:{}", ex);
return ResponseUtil.error("上传失败");
}
}

}

7. FilesManagementController 控制器

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
java复制代码package com.cyj.dream.file.controller.minio;

import cn.hutool.core.util.StrUtil;
import com.alibaba.fastjson.JSONObject;
import com.cyj.dream.core.aspect.annotation.ResponseResult;
import com.cyj.dream.core.constant.pagemodel.ResponseUtil;
import com.cyj.dream.file.model.FileUploadRecord;
import com.cyj.dream.file.service.FileManagementService;
import io.swagger.annotations.Api;
import io.swagger.annotations.ApiImplicitParam;
import io.swagger.annotations.ApiImplicitParams;
import io.swagger.annotations.ApiOperation;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;
import org.springframework.web.multipart.MultipartFile;

/**
* @Description: 文件管理
* @BelongsProject: DreamChardonnay
* @BelongsPackage: com.cyj.dream.file.controller.minio
* @Author: ChenYongJia
* @CreateTime: 2021-09-13
* @Email: chen87647213@163.com
* @Version: 1.0
*/
@Slf4j
@ResponseResult
@RestController
@RequestMapping(value = "/file/manager", name = "文件管理")
@Api(value = "/file/manager", tags = "文件管理")
public class FilesManagementController {

@Autowired
private FileManagementService fileManagementService;

/**
* 初始化大文件上传
*
* @author ChenYongJia
* @date 2021-9-13
*/
@ApiOperation("初始化大文件上传")
@ApiImplicitParams({
@ApiImplicitParam(name = "md5", value = "文件信息的md5值", dataType = "String", required = true),
@ApiImplicitParam(name = "chunkCount", value = "分片计数", dataType = "String", required = true),
@ApiImplicitParam(name = "fileSize", value = "文件大小", dataType = "String", required = true)
})
@RequestMapping(value = "/initChunkUpload", name = "初始化大文件上传", method = RequestMethod.POST)
public Object initChunkUpload(String md5, String chunkCount, String fileSize) {
log.info("进入 初始化大文件上传 控制器方法,md5:{},chunkCount:{},fileSize:{}", md5, chunkCount, fileSize);
if(StrUtil.isEmpty(md5) || StrUtil.isEmpty(chunkCount) || StrUtil.isEmpty(fileSize)){
log.error("参数缺失请检查参数后重新提交~");
return ResponseUtil.error("参数缺失请检查参数后重新提交");
}
return fileManagementService.initChunkUpload(md5, chunkCount, fileSize);
}

/**
* 合并文件并返回文件信息
*
* @author ChenYongJia
* @date 2021-9-13
*/
@ApiOperation("合并文件并返回文件信息")
@ApiImplicitParams({
@ApiImplicitParam(name = "md5", value = "文件信息的md5值", dataType = "String", required = true),
@ApiImplicitParam(name = "fileName", value = "文件名称", dataType = "String", required = true)
})
@RequestMapping(value = "/composeFile", name = "合并文件并返回文件信息", method = RequestMethod.POST)
public Object composeFile(String md5, String fileName) {
log.info("进入 合并文件并返回文件信息 控制器方法,md5:{},fileName:{}", md5, fileName);
if(StrUtil.isEmpty(md5) || StrUtil.isEmpty(fileName)){
log.error("参数缺失请检查参数后重新提交~");
return ResponseUtil.error("参数缺失请检查参数后重新提交");
}
return fileManagementService.composeFile(md5, fileName);
}

/**
* 限制上传直接上传
*
* @author ChenYongJia
* @date 2021-9-13
*/
@ApiOperation("限制上传直接上传")
@ApiImplicitParams({
@ApiImplicitParam(name = "file", value = "文件信息", dataType = "MultipartFile", required = true),
@ApiImplicitParam(name = "fileName", value = "文件名称", dataType = "String", required = true)
})
@RequestMapping(value = "/authUpload", name = "限制上传直接上传", method = RequestMethod.POST)
public Object authUpload(MultipartFile file, String fileName) {
log.info("进入 限制上传直接上传 控制器方法,fileSize:{},fileName:{}", file.getSize(), fileName);
if(file == null || file.getSize() == 0 || StrUtil.isEmpty(fileName)){
log.error("参数缺失请检查参数后重新提交~");
return ResponseUtil.error("参数缺失请检查参数后重新提交");
}
return fileManagementService.authUpload(file, fileName);
}

/**
* 放行上传直接上传

* @author ChenYongJia
* @date 2021-9-13
*/
@ApiOperation("放行上传直接上传")
@ApiImplicitParams({
@ApiImplicitParam(name = "file", value = "文件信息", dataType = "MultipartFile", required = true),
@ApiImplicitParam(name = "fileName", value = "文件名称", dataType = "String", required = true)
})
@RequestMapping(value = "/allowUpload", name = "放行上传直接上传", method = RequestMethod.POST)
public Object allowUpload(MultipartFile file, String fileName) {
log.info("进入 放行上传直接上传--如何测试-- 控制器方法,fileSize:{},fileName:{}", file.getSize(), fileName);
if(file == null || file.getSize() == 0 || StrUtil.isEmpty(fileName)){
log.error("参数缺失请检查参数后重新提交~");
return ResponseUtil.error("参数缺失请检查参数后重新提交");
}
return fileManagementService.allowUpload(file, fileName);
}

/**
* 获取到图片文件的base64
*
* @author ChenYongJia
* @date 2021-9-13
*/
@ApiOperation("获取到图片文件的base64")
@ApiImplicitParams({
@ApiImplicitParam(name = "fileUploadRecord", value = "文件上传记录对象", dataType = "FileUploadRecord", required = true)
})
@RequestMapping(value = "/getPicBase64", name = "获取到图片文件的base64", method = RequestMethod.POST)
public Object getPicBase64(@RequestBody FileUploadRecord fileUploadRecord) {
log.info("进入 获取到图片文件的base64 控制器方法,fileUploadRecord:{},", JSONObject.toJSONString(fileUploadRecord));
if(fileUploadRecord == null){
log.error("参数缺失请检查参数后重新提交~");
return ResponseUtil.error("参数缺失请检查参数后重新提交");
}
return fileManagementService.getPicBase64(fileUploadRecord);
}

}

8. 结果如下

1632966052.jpg

1632966053(1).jpg

最后感谢大家耐心观看完毕,原创不易,留个点赞收藏是您对我最大的鼓励!


🎉总结:

  • 更多参考精彩博文请看这里:《陈永佳的博客》
  • 喜欢博主的小伙伴可以加个关注、点个赞哦,持续更新嘿嘿!

本文转载自: 掘金

开发者博客 – 和开发相关的 这里全都有

0%