\u82f1\u6587\u9898\u76ee\uff1aBagging, Boosting, and C4.5<\/p>\n\n\n
\u4e2d\u6587\u9898\u76ee\uff1a\u81ea\u52a9\u805a\u96c6\u3001\u63d0\u5347\u548cC4.5<\/p>\n\n\n
\u4f5c\u8005\uff1aJ. R. Quinlan<\/p>\n\n\n
\u4e66\u7c4d\uff1aHandbook of statistics<\/p>\n\n\n
\u53d1\u8868\u65e5\u671f\uff1a2005<\/p>\n\n\n
\u5728\u673a\u5668\u5b66\u4e60\u9886\u57df\uff0c\u63d0\u5347\u6a21\u578b\u7684\u9884\u6d4b\u7cbe\u5ea6\u4e00\u76f4\u662f\u7814\u7a76\u4eba\u5458\u7684\u6838\u5fc3\u76ee\u6807\u4e4b\u4e00\u3002\u7531J. R. Quinlan\u64b0\u5199\u7684\u300aBagging, Boosting, and C4.5\u300b\u6df1\u5165\u63a2\u8ba8\u4e86\u4e24\u79cd\u5728\u5206\u7c7b\u5668\u5b66\u4e60\u7cfb\u7edf\u4e2d\u5907\u53d7\u5173\u6ce8\u7684\u6280\u672f\uff1a\u81ea\u52a9\u805a\u96c6\uff08Bagging\uff09\u548c\u63d0\u5347\uff08Boosting\uff09\u3002\u8fd9\u7bc7\u8bba\u6587\u4e0d\u4ec5\u6bd4\u8f83\u4e86\u8fd9\u4e24\u79cd\u65b9\u6cd5\u5728\u4e0d\u540c\u6570\u636e\u96c6\u4e0a\u7684\u8868\u73b0\uff0c\u8fd8\u63d0\u51fa\u4e86\u5bf9\u5176\u6548\u679c\u7684\u8be6\u7ec6\u5206\u6790\u3002\u901a\u8fc7\u5e94\u7528C4.5\u51b3\u7b56\u6811\u6a21\u578b\uff0c\u4f5c\u8005\u63ed\u793a\u4e86\u8fd9\u4e24\u79cd\u65b9\u6cd5\u5728\u63d0\u5347\u9884\u6d4b\u7cbe\u5ea6\u65b9\u9762\u7684\u6f5c\u529b\u53ca\u5176\u5c40\u9650\u6027\uff0c\u7279\u522b\u662f\u5f53Boosting\u5728\u67d0\u4e9b\u6570\u636e\u96c6\u4e0a\u53ef\u80fd\u5f15\u53d1\u7cbe\u5ea6\u7684\u4e0b\u964d\u65f6\u3002\u5bf9\u4e8e\u90a3\u4e9b\u6e34\u671b\u4e86\u89e3\u673a\u5668\u5b66\u4e60\u524d\u6cbf\u6280\u672f\u5e76\u5e0c\u671b\u63d0\u9ad8\u6a21\u578b\u6027\u80fd\u7684\u7814\u7a76\u4eba\u5458\u800c\u8a00\uff0c\u8fd9\u7bc7\u8bba\u6587\u65e0\u7591\u662f\u4e00\u4efd\u91cd\u8981\u7684\u53c2\u8003\u3002<\/p>\n\n\n\n
\u4ee5\u4e0b\u662f\u6839\u636e\u60a8\u63d0\u4f9b\u7684\u8bba\u6587\u5185\u5bb9\u603b\u7ed3\u51fa\u7684\u5404\u4e2a\u65b9\u9762\uff1a<\/p>\n\n\n
<\u7814\u7a76\u80cc\u666f\u4e0e\u76ee\u7684><\/p>\n\n\n
\u8fd1\u5e74\u6765\uff0c\u673a\u5668\u5b66\u4e60\u4e2d\u7684\u5206\u7c7b\u5668\u6027\u80fd\u63d0\u5347\u6210\u4e3a\u4e86\u7814\u7a76\u7684\u91cd\u70b9\u3002\u901a\u8fc7\u751f\u6210\u548c\u7ec4\u5408\u591a\u4e2a\u5206\u7c7b\u5668\uff0c\u81ea\u52a9\u805a\u96c6\uff08Bagging\uff09\u548c\u63d0\u5347\uff08Boosting\uff09\u6280\u672f\u5728\u63d0\u9ad8\u5206\u7c7b\u5668\u9884\u6d4b\u7cbe\u5ea6\u65b9\u9762\u5c55\u73b0\u4e86\u663e\u8457\u6f5c\u529b\u3002\u672c\u6587\u7684\u7814\u7a76\u76ee\u7684\u5728\u4e8e\u6bd4\u8f83\u8fd9\u4e24\u79cd\u65b9\u6cd5\u5728\u4e0d\u540c\u6570\u636e\u96c6\u4e0a\u7684\u8868\u73b0\uff0c\u5e76\u63a2\u8ba8\u5176\u5728\u63d0\u5347\u5206\u7c7b\u5668\u7cbe\u5ea6\u65b9\u9762\u7684\u6709\u6548\u6027\u53ca\u5176\u6f5c\u5728\u7684\u5c40\u9650\u6027\u3002<\/p>\n\n\n
<\u521b\u65b0\u70b9><\/p>\n\n\n
\u672c\u6587\u521b\u65b0\u6027\u5730\u5c06Bagging\u548cBoosting\u5e94\u7528\u4e8eC4.5\u51b3\u7b56\u6811\u7b97\u6cd5\uff0c\u5168\u9762\u5206\u6790\u4e86\u8fd9\u4e24\u79cd\u65b9\u6cd5\u5728\u5b9e\u9645\u6570\u636e\u96c6\u4e0a\u7684\u8868\u73b0\u3002\u540c\u65f6\uff0c\u4f5c\u8005\u5bf9Boosting\u65b9\u6cd5\u8fdb\u884c\u4e86\u6539\u8fdb\uff0c\u901a\u8fc7\u8c03\u6574\u5206\u7c7b\u5668\u7684\u6295\u7968\u6743\u91cd\u6765\u964d\u4f4e\u5176\u5728\u67d0\u4e9b\u6570\u636e\u96c6\u4e0a\u8868\u73b0\u4e0d\u4f73\u7684\u98ce\u9669\uff0c\u5e76\u5b9e\u73b0\u4e86\u66f4\u597d\u7684\u9884\u6d4b\u7ed3\u679c\u3002<\/p>\n\n\n
<\u7ed3\u8bba><\/p>\n\n\n
\u7814\u7a76\u8868\u660e\uff0cBagging\u548cBoosting\u90fd\u80fd\u663e\u8457\u63d0\u9ad8\u5206\u7c7b\u5668\u7684\u9884\u6d4b\u7cbe\u5ea6\u3002Bagging\u5728\u7a33\u5b9a\u6027\u4e0a\u8868\u73b0\u66f4\u597d\uff0c\u800cBoosting\u867d\u7136\u5728\u591a\u6570\u60c5\u51b5\u4e0b\u6548\u679c\u66f4\u4f73\uff0c\u4f46\u5728\u67d0\u4e9b\u6570\u636e\u96c6\u4e0a\u53ef\u80fd\u4f1a\u5bfc\u81f4\u9884\u6d4b\u7cbe\u5ea6\u7684\u4e0b\u964d\u3002<\/strong>\u901a\u8fc7\u4fee\u6539Boosting\u7684\u6295\u7968\u673a\u5236\uff0c\u53ef\u4ee5\u8fdb\u4e00\u6b65\u63d0\u9ad8\u5176\u5728\u5927\u591a\u6570\u6570\u636e\u96c6\u4e0a\u7684\u8868\u73b0\u3002<\/p>\n\n\n <\u5b9e\u9a8c\u5185\u5bb9><\/p>\n\n\n \u672c\u6587\u8fdb\u884c\u4e86\u5927\u91cf\u5b9e\u9a8c\uff0c\u91c7\u7528\u4e86UCI\u673a\u5668\u5b66\u4e60\u6570\u636e\u5e93\u4e2d\u7684\u591a\u4e2a\u6570\u636e\u96c6\uff0c\u5e76\u901a\u8fc710\u6b2110\u6298\u4ea4\u53c9\u9a8c\u8bc1\u8bc4\u4f30\u4e86Bagging\u548cBoosting\u5728C4.5\u51b3\u7b56\u6811\u4e0a\u7684\u6548\u679c\u3002\u5b9e\u9a8c\u7ed3\u679c\u5c55\u793a\u4e86\u8fd9\u4e24\u79cd\u65b9\u6cd5\u5728\u4e0d\u540c\u6570\u636e\u96c6\u4e0a\u7684\u5177\u4f53\u8868\u73b0\uff0c\u5e76\u8be6\u7ec6\u5206\u6790\u4e86Boosting\u5728\u67d0\u4e9b\u60c5\u51b5\u4e0b\u53ef\u80fd\u5bfc\u81f4\u6027\u80fd\u4e0b\u964d\u7684\u539f\u56e0\u3002<\/p>\n\n\n <\u5bf9\u672c\u9886\u57df\u7684\u8d21\u732e><\/p>\n\n\n \u672c\u6587\u5bf9\u673a\u5668\u5b66\u4e60\u9886\u57df\u7684\u8d21\u732e\u5728\u4e8e\uff0c\u901a\u8fc7\u8be6\u7ec6\u7684\u5b9e\u9a8c\u548c\u5206\u6790\uff0c\u9a8c\u8bc1\u4e86Bagging\u548cBoosting\u5728\u63d0\u5347\u5206\u7c7b\u5668\u7cbe\u5ea6\u65b9\u9762\u7684\u6709\u6548\u6027\uff0c\u5e76\u63d0\u51fa\u4e86\u5bf9Boosting\u65b9\u6cd5\u7684\u6539\u8fdb\u5efa\u8bae\uff0c\u8fd9\u4e3a\u540e\u7eed\u7684\u7814\u7a76\u548c\u5e94\u7528\u63d0\u4f9b\u4e86\u91cd\u8981\u7684\u53c2\u8003\u3002<\/p>\n\n\n <\u5b58\u5728\u7684\u4e0d\u8db3><\/p>\n\n\n \u5c3d\u7ba1Boosting\u5728\u591a\u6570\u60c5\u51b5\u4e0b\u8868\u73b0\u4f18\u5f02\uff0c\u4f46\u5176\u5bf9\u67d0\u4e9b\u6570\u636e\u96c6\u7684\u7cbe\u5ea6\u63d0\u5347\u4e0d\u660e\u663e\uff0c\u751a\u81f3\u53ef\u80fd\u5bfc\u81f4\u6027\u80fd\u7684\u4e0b\u964d\u3002\u672c\u6587\u867d\u7136\u63d0\u51fa\u4e86\u4fee\u6539Boosting\u6295\u7968\u6743\u91cd\u7684\u65b9\u6cd5\uff0c\u4f46\u8fd9\u4e00\u65b9\u6cd5\u8f83\u4e3a\u76f4\u89c2\uff0c\u7f3a\u4e4f\u4e25\u5bc6\u7684\u7406\u8bba\u652f\u6301\u3002<\/p>\n\n\n <\u672a\u6765\u7684\u5de5\u4f5c><\/p>\n\n\n \u672a\u6765\u7684\u5de5\u4f5c\u65b9\u5411\u5305\u62ec\u8fdb\u4e00\u6b65\u5b8c\u5584Boosting\u65b9\u6cd5\uff0c\u7279\u522b\u662f\u5728\u6295\u7968\u673a\u5236\u65b9\u9762\u7684\u6539\u8fdb\uff0c\u5c1d\u8bd5\u5f00\u53d1\u5177\u6709\u66f4\u5f3a\u7406\u8bba\u57fa\u7840\u7684\u6295\u7968\u6743\u91cd\u8c03\u6574\u65b9\u6cd5\u3002\u6b64\u5916\uff0c\u7814\u7a76\u5982\u4f55\u5728\u4e0d\u540c\u7c7b\u578b\u7684\u6570\u636e\u96c6\u4e0a\u66f4\u597d\u5730\u5e73\u8861Bagging\u548cBoosting\u7684\u4f18\u7f3a\u70b9\u4e5f\u662f\u4e00\u4e2a\u91cd\u8981\u7684\u65b9\u5411\u3002<\/p>\n\n\n \u201cDesigners of empirical machine learning systems are concerned with such issues as the computational cost of the learning method and the accuracy and intelligibility of the theories that it constructs. Much of the research in learning has tended to focus on improved predictive accuracy, so that the performance of new systems is often reported from this perspective.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n \u7ecf\u9a8c\u6027\u673a\u5668\u5b66\u4e60\u7cfb\u7edf\u7684\u8bbe\u8ba1\u8005\u5173\u6ce8\u7684\u95ee\u9898\u5305\u62ec\u5b66\u4e60\u65b9\u6cd5\u7684\u8ba1\u7b97\u6210\u672c\u4ee5\u53ca\u6240\u6784\u5efa\u7406\u8bba\u7684\u51c6\u786e\u6027\u548c\u53ef\u7406\u89e3\u6027\u3002\u8bb8\u591a\u5b66\u4e60\u7814\u7a76\u5f80\u5f80\u96c6\u4e2d\u4e8e\u63d0\u9ad8\u9884\u6d4b\u51c6\u786e\u6027\uff0c\u56e0\u6b64\u65b0\u7cfb\u7edf\u7684\u6027\u80fd\u901a\u5e38\u4ece\u8fd9\u4e00\u89d2\u5ea6\u8fdb\u884c\u62a5\u544a\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u8fd9\u6bb5\u8bdd\u5f3a\u8c03\u4e86\u5728\u673a\u5668\u5b66\u4e60\u9886\u57df\uff0c\u7814\u7a76\u4eba\u5458\u5bf9\u6a21\u578b\u6027\u80fd\u7684\u5173\u6ce8\u70b9\u4e3b\u8981\u96c6\u4e2d\u5728\u9884\u6d4b\u51c6\u786e\u6027\u4e0a\u3002\u8fd9\u8868\u660e\u63d0\u9ad8\u6a21\u578b\u7684\u9884\u6d4b\u7cbe\u5ea6\u662f\u63a8\u52a8\u65b0\u6280\u672f\u53d1\u5c55\u7684\u6838\u5fc3\u9a71\u52a8\u529b\uff0c\u4e5f\u4e3aBagging\u548cBoosting\u6280\u672f\u7684\u53d1\u5c55\u63d0\u4f9b\u4e86\u7814\u7a76\u80cc\u666f\u3002<\/p>\n\n\n \u201cThere has recently been renewed interest in increasing accuracy by generating and aggregating multiple classifiers. Although the idea of growing multiple trees is not new, the justification for such methods is often empirical. In contrast, two new approaches for producing and using several classifiers are applicable to a wide variety of learning systems and are based on theoretical analyses of the behavior of the composite classifier.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n \u6700\u8fd1\uff0c\u4eba\u4eec\u91cd\u65b0\u5173\u6ce8\u901a\u8fc7\u751f\u6210\u548c\u805a\u5408\u591a\u4e2a\u5206\u7c7b\u5668\u6765\u63d0\u9ad8\u51c6\u786e\u6027\u3002\u867d\u7136\u751f\u6210\u591a\u4e2a\u6811\u7684\u60f3\u6cd5\u5e76\u4e0d\u65b0\u9c9c\uff0c\u4f46\u8fd9\u79cd\u65b9\u6cd5\u7684\u5408\u7406\u6027\u901a\u5e38\u662f\u7ecf\u9a8c\u6027\u7684\u3002\u76f8\u6bd4\u4e4b\u4e0b\uff0c\u4e24\u79cd\u65b0\u65b9\u6cd5\uff08Bagging\u548cBoosting\uff09\u53ef\u4ee5\u5e94\u7528\u4e8e\u591a\u79cd\u5b66\u4e60\u7cfb\u7edf\uff0c\u5e76\u57fa\u4e8e\u5bf9\u7ec4\u5408\u5206\u7c7b\u5668\u884c\u4e3a\u7684\u7406\u8bba\u5206\u6790\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u8fd9\u6bb5\u6587\u5b57\u8bf4\u660e\u4e86Bagging\u548cBoosting\u65b9\u6cd5\u7684\u7406\u8bba\u57fa\u7840\uff0c\u4ee5\u53ca\u5b83\u4eec\u5982\u4f55\u5728\u63d0\u9ad8\u5206\u7c7b\u5668\u51c6\u786e\u6027\u65b9\u9762\u5f97\u5230\u4e86\u5e7f\u6cdb\u5e94\u7528\u3002\u4f5c\u8005\u5c06\u8fd9\u4e24\u79cd\u65b9\u6cd5\u4e0e\u4f20\u7edf\u7684\u751f\u6210\u591a\u68f5\u6811\u7684\u7b56\u7565\u8fdb\u884c\u4e86\u5bf9\u6bd4\uff0c\u7a81\u51fa\u4e86\u5b83\u4eec\u5728\u7406\u8bba\u4e0a\u66f4\u4e3a\u575a\u5b9e\u7684\u57fa\u7840\u3002<\/p>\n\n\n \u201cBoth bootstrap aggregating or bagging and boosting manipulate the training data in order to generate different classifiers. Bagging produces replicate training sets by sampling with replacement from the training instances, and boosting uses all instances at each repetition, but maintains a weight for each instance in the training set that reflects its importance.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n \u81ea\u52a9\u805a\u96c6\uff08Bagging\uff09\u548c\u63d0\u5347\uff08Boosting\uff09\u90fd\u901a\u8fc7\u64cd\u63a7\u8bad\u7ec3\u6570\u636e\u6765\u751f\u6210\u4e0d\u540c\u7684\u5206\u7c7b\u5668\u3002Bagging\u901a\u8fc7\u4ece\u8bad\u7ec3\u5b9e\u4f8b\u4e2d\u6709\u653e\u56de\u62bd\u6837\u6765\u751f\u6210\u91cd\u590d\u7684\u8bad\u7ec3\u96c6\uff0c\u800cBoosting\u5219\u5728\u6bcf\u6b21\u91cd\u590d\u4e2d\u4f7f\u7528\u6240\u6709\u5b9e\u4f8b\uff0c\u4f46\u4e3a\u8bad\u7ec3\u96c6\u4e2d\u6bcf\u4e2a\u5b9e\u4f8b\u4fdd\u6301\u4e00\u4e2a\u53cd\u6620\u5176\u91cd\u8981\u6027\u7684\u6743\u91cd\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u8fd9\u6bb5\u8bdd\u89e3\u91ca\u4e86Bagging\u548cBoosting\u5728\u751f\u6210\u591a\u4e2a\u5206\u7c7b\u5668\u65f6\u6240\u4f7f\u7528\u7684\u4e0d\u540c\u6570\u636e\u5904\u7406\u65b9\u6cd5\u3002Bagging\u91c7\u7528\u968f\u673a\u62bd\u6837\u7684\u65b9\u5f0f\uff0c\u800cBoosting\u901a\u8fc7\u8c03\u6574\u5b9e\u4f8b\u6743\u91cd\u6765\u5f15\u5bfc\u6a21\u578b\u5b66\u4e60\u4e0d\u540c\u7684\u7279\u5f81\u3002\u4f5c\u8005\u6e05\u6670\u5730\u63cf\u8ff0\u4e86\u4e24\u8005\u7684\u5de5\u4f5c\u673a\u5236\uff0c\u4e3a\u8bfb\u8005\u7406\u89e3\u8fd9\u4e9b\u6280\u672f\u7684\u6838\u5fc3\u539f\u7406\u5960\u5b9a\u4e86\u57fa\u7840\u3002<\/p>\n\n\n \u201cThe final classifier also aggregates the learned classifiers by voting, but each classifier\u2019s vote is a function of its accuracy.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n \u6700\u7ec8\u7684\u5206\u7c7b\u5668\u8fd8\u901a\u8fc7\u6295\u7968\u6765\u805a\u5408\u5b66\u4e60\u5230\u7684\u5206\u7c7b\u5668\uff0c\u4f46\u6bcf\u4e2a\u5206\u7c7b\u5668\u7684\u6295\u7968\u6743\u91cd\u662f\u5176\u51c6\u786e\u6027\u7684\u51fd\u6570\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u8fd9\u53e5\u8bdd\u8bf4\u660e\u4e86Boosting\u65b9\u6cd5\u4e2d\u6295\u7968\u673a\u5236\u7684\u91cd\u8981\u6027\uff0c\u7279\u522b\u662f\u5728\u6700\u7ec8\u51b3\u7b56\u65f6\u5982\u4f55\u4f9d\u636e\u6bcf\u4e2a\u5206\u7c7b\u5668\u7684\u51c6\u786e\u6027\u6765\u5206\u914d\u6743\u91cd\u3002\u4e0eBagging\u7684\u7b49\u6743\u91cd\u6295\u7968\u4e0d\u540c\uff0cBoosting\u66f4\u4e3a\u590d\u6742\uff0c\u80fd\u591f\u66f4\u597d\u5730\u53d1\u6325\u6bcf\u4e2a\u5206\u7c7b\u5668\u7684\u4f18\u52bf\u3002<\/p>\n\n\n \u603b\u7ed3\u8bc4\u8bba<\/p>\n\n\n \u5f15\u8a00\u90e8\u5206\u901a\u8fc7\u5bf9\u673a\u5668\u5b66\u4e60\u7cfb\u7edf\u8bbe\u8ba1\u8005\u7684\u5173\u6ce8\u70b9\u7684\u8ba8\u8bba\uff0c\u4e3aBagging\u548cBoosting\u6280\u672f\u7684\u63d0\u51fa\u63d0\u4f9b\u4e86\u80cc\u666f\uff0c\u5e76\u4e14\u8be6\u7ec6\u89e3\u91ca\u4e86\u8fd9\u4e24\u79cd\u65b9\u6cd5\u7684\u7406\u8bba\u57fa\u7840\u548c\u64cd\u4f5c\u673a\u5236\u3002\u8fd9\u4e9b\u539f\u6587\u6bb5\u843d\u6709\u52a9\u4e8e\u7406\u89e3\u4f5c\u8005\u4e3a\u4f55\u5c06\u91cd\u70b9\u653e\u5728\u8fd9\u4e24\u79cd\u65b9\u6cd5\u4e0a\uff0c\u5e76\u5c55\u793a\u4e86\u5b83\u4eec\u5728\u673a\u5668\u5b66\u4e60\u9886\u57df\u4e2d\u7684\u72ec\u7279\u4ef7\u503c\u3002\u901a\u8fc7\u8fd9\u4e9b\u8ba8\u8bba\uff0c\u4f5c\u8005\u4e3a\u540e\u7eed\u7ae0\u8282\u7684\u5b9e\u9a8c\u548c\u5206\u6790\u5960\u5b9a\u4e86\u7406\u8bba\u57fa\u7840\uff0c\u4e5f\u4e3a\u8bfb\u8005\u7406\u89e3\u8fd9\u4e9b\u65b9\u6cd5\u7684\u4f18\u52bf\u548c\u6311\u6218\u63d0\u4f9b\u4e86\u5fc5\u8981\u7684\u80cc\u666f\u4fe1\u606f\u3002<\/p>\n\n\n \u201cBagging produces replicate training sets by sampling with replacement from the original instances. This training set is the same size as the original data, but some instances may not appear in it while others appear more than once.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n Bagging\u901a\u8fc7\u4ece\u539f\u59cb\u5b9e\u4f8b\u4e2d\u6709\u653e\u56de\u62bd\u6837\u6765\u751f\u6210\u91cd\u590d\u7684\u8bad\u7ec3\u96c6\u3002\u8fd9\u4e2a\u8bad\u7ec3\u96c6\u7684\u5927\u5c0f\u4e0e\u539f\u59cb\u6570\u636e\u76f8\u540c\uff0c\u4f46\u67d0\u4e9b\u5b9e\u4f8b\u53ef\u80fd\u4e0d\u4f1a\u51fa\u73b0\u5728\u5176\u4e2d\uff0c\u800c\u5176\u4ed6\u5b9e\u4f8b\u5219\u53ef\u80fd\u51fa\u73b0\u591a\u6b21\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u8fd9\u6bb5\u8bdd\u6e05\u695a\u5730\u63cf\u8ff0\u4e86Bagging\u65b9\u6cd5\u751f\u6210\u8bad\u7ec3\u96c6\u7684\u8fc7\u7a0b\u3002\u901a\u8fc7\u6709\u653e\u56de\u62bd\u6837\uff0cBagging\u80fd\u591f\u5728\u4e0d\u589e\u52a0\u6570\u636e\u96c6\u5927\u5c0f\u7684\u60c5\u51b5\u4e0b\u751f\u6210\u591a\u6837\u5316\u7684\u8bad\u7ec3\u96c6\uff0c\u4ece\u800c\u5728\u6a21\u578b\u8bad\u7ec3\u4e2d\u5f15\u5165\u53d8\u5f02\u6027\u3002\u8fd9\u4e00\u8fc7\u7a0b\u6709\u52a9\u4e8e\u63d0\u9ad8\u6a21\u578b\u7684\u7a33\u5b9a\u6027\uff0c\u5c24\u5176\u662f\u5728\u9762\u5bf9\u4e0d\u7a33\u5b9a\u7684\u5b66\u4e60\u7b97\u6cd5\u65f6\u3002<\/p>\n\n\n \u201cThe final classifier is formed by aggregating the classifiers from these trials. To classify an instance, a vote for each class is recorded by every classifier, and the class with the most votes becomes the final prediction.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n \u6700\u7ec8\u7684\u5206\u7c7b\u5668\u662f\u901a\u8fc7\u805a\u5408\u8fd9\u4e9b\u8bd5\u9a8c\u4e2d\u7684\u5206\u7c7b\u5668\u5f62\u6210\u7684\u3002\u4e3a\u4e86\u5bf9\u4e00\u4e2a\u5b9e\u4f8b\u8fdb\u884c\u5206\u7c7b\uff0c\u6bcf\u4e2a\u5206\u7c7b\u5668\u4e3a\u6bcf\u4e2a\u7c7b\u522b\u6295\u7968\uff0c\u5f97\u7968\u6700\u591a\u7684\u7c7b\u522b\u6210\u4e3a\u6700\u7ec8\u7684\u9884\u6d4b\u7ed3\u679c\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u8fd9\u6bb5\u8bdd\u89e3\u91ca\u4e86Bagging\u7684\u6295\u7968\u673a\u5236\uff0c\u5373\u901a\u8fc7\u591a\u4e2a\u5206\u7c7b\u5668\u7684\u6295\u7968\u6765\u51b3\u5b9a\u6700\u7ec8\u7684\u5206\u7c7b\u7ed3\u679c\u3002Bagging\u7684\u8fd9\u79cd\u7b49\u6743\u91cd\u6295\u7968\u65b9\u5f0f\u5728\u591a\u4e2a\u6a21\u578b\u4e4b\u95f4\u53d6\u957f\u8865\u77ed\uff0c\u4ece\u800c\u63d0\u9ad8\u6574\u4f53\u5206\u7c7b\u5668\u7684\u6027\u80fd\u3002\u8fd9\u79cd\u65b9\u6cd5\u7b80\u5355\u5374\u6709\u6548\uff0c\u5728\u5b9e\u9645\u5e94\u7528\u4e2d\u8868\u73b0\u51fa\u8272\u3002<\/p>\n\n\n \u201cInstead of drawing a succession of independent bootstrap samples from the original instances, boosting maintains a weight for each instance; the higher the weight, the more the instance influences the classifier learned.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n Boosting\u5e76\u4e0d\u662f\u4ece\u539f\u59cb\u5b9e\u4f8b\u4e2d\u62bd\u53d6\u4e00\u7cfb\u5217\u72ec\u7acb\u7684\u81ea\u52a9\u6837\u672c\uff0c\u800c\u662f\u4e3a\u6bcf\u4e2a\u5b9e\u4f8b\u7ef4\u62a4\u4e00\u4e2a\u6743\u91cd\uff1b\u6743\u91cd\u8d8a\u9ad8\uff0c\u5b9e\u4f8b\u5bf9\u6240\u5b66\u4e60\u5206\u7c7b\u5668\u7684\u5f71\u54cd\u5c31\u8d8a\u5927\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u8fd9\u6bb5\u8bdd\u4ecb\u7ecd\u4e86Boosting\u4e0eBagging\u5728\u6570\u636e\u5904\u7406\u4e0a\u7684\u5173\u952e\u533a\u522b\u3002Boosting\u901a\u8fc7\u8c03\u6574\u6bcf\u4e2a\u5b9e\u4f8b\u7684\u6743\u91cd\u6765\u5f15\u5bfc\u6a21\u578b\u5173\u6ce8\u90a3\u4e9b\u96be\u4ee5\u5206\u7c7b\u7684\u5b9e\u4f8b\uff0c\u4ece\u800c\u9010\u6b65\u63d0\u9ad8\u5206\u7c7b\u5668\u7684\u6027\u80fd\u3002\u8fd9\u79cd\u6743\u91cd\u8c03\u6574\u673a\u5236\u4f7f\u5f97Boosting\u5728\u5904\u7406\u590d\u6742\u6570\u636e\u65f6\u80fd\u591f\u66f4\u7075\u6d3b\u5730\u5e94\u5bf9\u4e0d\u540c\u7684\u5206\u7c7b\u6311\u6218\u3002<\/p>\n\n\n \u201cAt each trial, the vector of weights is adjusted to reflect the performance of the corresponding classifier, with the result that the weight of misclassified instances is increased.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n \u5728\u6bcf\u6b21\u8bd5\u9a8c\u4e2d\uff0c\u6743\u91cd\u5411\u91cf\u4f1a\u6839\u636e\u76f8\u5e94\u5206\u7c7b\u5668\u7684\u8868\u73b0\u8fdb\u884c\u8c03\u6574\uff0c\u7ed3\u679c\u662f\u88ab\u9519\u8bef\u5206\u7c7b\u7684\u5b9e\u4f8b\u7684\u6743\u91cd\u4f1a\u589e\u52a0\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u8fd9\u6bb5\u8bdd\u63ed\u793a\u4e86Boosting\u7b97\u6cd5\u7684\u6838\u5fc3\u673a\u5236\uff1a\u901a\u8fc7\u589e\u5927\u88ab\u9519\u8bef\u5206\u7c7b\u7684\u5b9e\u4f8b\u7684\u6743\u91cd\uff0cBoosting\u65b9\u6cd5\u8feb\u4f7f\u540e\u7eed\u5206\u7c7b\u5668\u66f4\u52a0\u5173\u6ce8\u8fd9\u4e9b\u96be\u4ee5\u5206\u7c7b\u7684\u5b9e\u4f8b\u3002\u8fd9\u79cd\u52a8\u6001\u8c03\u6574\u6743\u91cd\u7684\u65b9\u6cd5\u80fd\u591f\u6709\u6548\u63d0\u9ad8\u5206\u7c7b\u5668\u7684\u6574\u4f53\u6027\u80fd\uff0c\u7279\u522b\u662f\u5728\u5904\u7406\u504f\u659c\u6570\u636e\u6216\u4e0d\u5e73\u8861\u6570\u636e\u96c6\u65f6\u8868\u73b0\u4f18\u5f02\u3002<\/p>\n\n\n \u201cThe final classifier also aggregates the learned classifiers by voting, but each classifier\u2019s vote is a function of its accuracy.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n \u6700\u7ec8\u7684\u5206\u7c7b\u5668\u901a\u8fc7\u6295\u7968\u6765\u805a\u5408\u5df2\u5b66\u4e60\u5230\u7684\u5206\u7c7b\u5668\uff0c\u4f46\u6bcf\u4e2a\u5206\u7c7b\u5668\u7684\u6295\u7968\u6743\u91cd\u662f\u5176\u51c6\u786e\u6027\u7684\u51fd\u6570\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u8fd9\u6bb5\u8bdd\u8fdb\u4e00\u6b65\u89e3\u91ca\u4e86Boosting\u65b9\u6cd5\u4e2d\u7684\u6295\u7968\u673a\u5236\uff0c\u5f3a\u8c03\u4e86\u4e0eBagging\u4e0d\u540c\uff0cBoosting\u6839\u636e\u6bcf\u4e2a\u5206\u7c7b\u5668\u7684\u51c6\u786e\u6027\u6765\u8c03\u6574\u5176\u6295\u7968\u6743\u91cd\u3002\u8fd9\u4e00\u673a\u5236\u4f7f\u5f97Boosting\u5728\u805a\u5408\u591a\u4e2a\u5206\u7c7b\u5668\u65f6\u66f4\u52a0\u6ce8\u91cd\u9ad8\u8d28\u91cf\u7684\u5206\u7c7b\u5668\uff0c\u4ece\u800c\u63d0\u9ad8\u6700\u7ec8\u6a21\u578b\u7684\u6574\u4f53\u6027\u80fd\u3002<\/p>\n\n\n \u603b\u7ed3\u8bc4\u8bba<\/p>\n\n\n \u201cBagging and Boosting\u201d\u7ae0\u8282\u8be6\u7ec6\u63cf\u8ff0\u4e86\u8fd9\u4e24\u79cd\u65b9\u6cd5\u7684\u6838\u5fc3\u5de5\u4f5c\u539f\u7406\u548c\u64cd\u4f5c\u6b65\u9aa4\u3002Bagging\u901a\u8fc7\u6709\u653e\u56de\u62bd\u6837\u548c\u7b49\u6743\u91cd\u6295\u7968\u6765\u751f\u6210\u7a33\u5b9a\u7684\u5206\u7c7b\u5668\uff0c\u9002\u7528\u4e8e\u4e0d\u7a33\u5b9a\u7684\u5b66\u4e60\u7b97\u6cd5\uff1b\u800cBoosting\u901a\u8fc7\u52a8\u6001\u8c03\u6574\u5b9e\u4f8b\u6743\u91cd\u548c\u7cbe\u7ec6\u7684\u6295\u7968\u673a\u5236\uff0c\u9010\u6b65\u63d0\u5347\u5206\u7c7b\u5668\u7684\u6027\u80fd\uff0c\u7279\u522b\u662f\u5728\u590d\u6742\u6570\u636e\u96c6\u4e0a\u7684\u8868\u73b0\u5c24\u4e3a\u51fa\u8272\u3002\u8fd9\u4e9b\u63cf\u8ff0\u4e0d\u4ec5\u6709\u52a9\u4e8e\u7406\u89e3\u4e24\u79cd\u65b9\u6cd5\u7684\u57fa\u672c\u64cd\u4f5c\uff0c\u4e5f\u4e3a\u540e\u7eed\u7684\u5b9e\u9a8c\u90e8\u5206\u548c\u6027\u80fd\u6bd4\u8f83\u63d0\u4f9b\u4e86\u5fc5\u8981\u7684\u7406\u8bba\u80cc\u666f\u3002<\/p>\n\n\n \u201cThe experiments reported here were carried out on a representative collection of datasets from the UCI Machine Learning Repository. The 27 datasets, summarized in the Appendix, show considerable diversity in size, number of classes, and number and type of attributes.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n \u672c\u6587\u6240\u62a5\u544a\u7684\u5b9e\u9a8c\u662f\u5728UCI\u673a\u5668\u5b66\u4e60\u5e93\u4e2d\u4e00\u4e2a\u5177\u6709\u4ee3\u8868\u6027\u7684\u6570\u636e\u96c6\u96c6\u5408\u4e0a\u8fdb\u884c\u7684\u3002\u8fd927\u4e2a\u6570\u636e\u96c6\u7684\u5927\u5c0f\u3001\u7c7b\u522b\u6570\u91cf\u4ee5\u53ca\u5c5e\u6027\u7684\u6570\u91cf\u548c\u7c7b\u578b\u90fd\u663e\u793a\u51fa\u76f8\u5f53\u5927\u7684\u591a\u6837\u6027\uff0c\u8fd9\u4e9b\u4fe1\u606f\u5728\u9644\u5f55\u4e2d\u8fdb\u884c\u4e86\u603b\u7ed3\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u8fd9\u6bb5\u8bdd\u8bf4\u660e\u4e86\u5b9e\u9a8c\u7684\u5e7f\u6cdb\u6027\u548c\u4ee3\u8868\u6027\uff0c\u5b9e\u9a8c\u6570\u636e\u96c6\u6765\u81ea\u77e5\u540d\u7684UCI\u673a\u5668\u5b66\u4e60\u5e93\uff0c\u6db5\u76d6\u4e86\u4e0d\u540c\u7c7b\u578b\u7684\u6570\u636e\u3002\u8fd9\u79cd\u591a\u6837\u5316\u7684\u9009\u62e9\u4f7f\u5f97\u5b9e\u9a8c\u7ed3\u679c\u5177\u6709\u66f4\u5e7f\u6cdb\u7684\u9002\u7528\u6027\u548c\u8bf4\u670d\u529b\uff0c\u6709\u52a9\u4e8e\u9a8c\u8bc1Bagging\u548cBoosting\u5728\u5404\u79cd\u5b9e\u9645\u5e94\u7528\u4e2d\u7684\u6709\u6548\u6027\u3002<\/p>\n\n\n \u201cThe parameter T governing the number of classifiers generated was set at 50 for these experiments. Breiman (1996) notes that most of the improvement from bagging is evident within ten replications, and it is interesting to see the performance improvement that can be bought by a single order of magnitude increase in computation.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n \u5728\u8fd9\u4e9b\u5b9e\u9a8c\u4e2d\uff0c\u751f\u6210\u5206\u7c7b\u5668\u6570\u91cf\u7684\u53c2\u6570T\u88ab\u8bbe\u5b9a\u4e3a50\u3002Breiman\uff081996\uff09\u6307\u51fa\uff0c\u81ea\u52a9\u805a\u96c6\u7684\u5927\u90e8\u5206\u6539\u8fdb\u6548\u679c\u572810\u6b21\u91cd\u590d\u5185\u5c31\u5df2\u7ecf\u663e\u73b0\u51fa\u6765\uff0c\u56e0\u6b64\u89c2\u5bdf\u8ba1\u7b97\u91cf\u589e\u52a0\u4e00\u4e2a\u6570\u91cf\u7ea7\u6240\u5e26\u6765\u7684\u6027\u80fd\u63d0\u5347\u662f\u5f88\u6709\u8da3\u7684\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u4f5c\u8005\u901a\u8fc7\u5f15\u7528\u524d\u4eba\u7684\u7814\u7a76\uff0c\u89e3\u91ca\u4e86\u9009\u62e9T=50\u8fd9\u4e00\u53c2\u6570\u7684\u5408\u7406\u6027\uff0c\u5e76\u63d0\u51fa\u4e86\u5bf9\u8ba1\u7b97\u8d44\u6e90\u4e0e\u6027\u80fd\u63d0\u5347\u5173\u7cfb\u7684\u5174\u8da3\u70b9\u3002\u8fd9\u79cd\u8bbe\u7f6e\u65e2\u57fa\u4e8e\u7406\u8bba\u7814\u7a76\uff0c\u53c8\u4e3a\u8fdb\u4e00\u6b65\u63a2\u7d22Bagging\u548cBoosting\u7684\u6027\u80fd\u6f5c\u529b\u63d0\u4f9b\u4e86\u5b9e\u9645\u4f9d\u636e\u3002<\/p>\n\n\n \u201cIt is clear that, over these 27 datasets, both bagging and boosting lead to markedly more accurate classifiers. Bagging reduces C4.5\u2019s classification error by approximately 7.5% on average and is superior to C4.5 on 22 of the 27 datasets. Boosting reduces error by 9% on average, improves performance on 24 datasets, but degrades performance on six.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n \u5f88\u660e\u663e\uff0c\u5728\u8fd927\u4e2a\u6570\u636e\u96c6\u4e0a\uff0cBagging\u548cBoosting\u90fd\u663e\u8457\u63d0\u9ad8\u4e86\u5206\u7c7b\u5668\u7684\u51c6\u786e\u6027\u3002Bagging\u5e73\u5747\u5c06C4.5\u7684\u5206\u7c7b\u9519\u8bef\u7387\u964d\u4f4e\u4e86\u7ea67.5%\uff0c\u572827\u4e2a\u6570\u636e\u96c6\u4e2d\u670922\u4e2a\u4f18\u4e8eC4.5\u3002\u800cBoosting\u5e73\u5747\u5c06\u9519\u8bef\u7387\u964d\u4f4e\u4e869%\uff0c\u572824\u4e2a\u6570\u636e\u96c6\u4e0a\u63d0\u9ad8\u4e86\u6027\u80fd\uff0c\u4f46\u57286\u4e2a\u6570\u636e\u96c6\u4e0a\u5374\u5bfc\u81f4\u4e86\u6027\u80fd\u4e0b\u964d\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u8fd9\u6bb5\u8bdd\u603b\u7ed3\u4e86\u5b9e\u9a8c\u7684\u5173\u952e\u7ed3\u679c\uff0c\u8868\u660eBagging\u548cBoosting\u90fd\u80fd\u6709\u6548\u5730\u63d0\u9ad8\u5206\u7c7b\u5668\u6027\u80fd\uff0c\u5c3d\u7ba1Boosting\u5728\u67d0\u4e9b\u60c5\u51b5\u4e0b\u53ef\u80fd\u5f15\u53d1\u8d1f\u9762\u6548\u679c\u3002\u8fd9\u4e3a\u540e\u7eed\u7684\u6df1\u5165\u5206\u6790\u63d0\u4f9b\u4e86\u6570\u636e\u652f\u6301\uff0c\u4e5f\u6307\u51fa\u4e86Boosting\u65b9\u6cd5\u7684\u4e00\u4e9b\u6f5c\u5728\u98ce\u9669\u3002<\/p>\n\n\n \u201cWhen bagging and boosting are compared head to head, boosting leads to greater reduction in error and is superior to bagging on 19 of the 27 datasets, significant at the 0.01 level. However, the effect of boosting is more erratic, leading to significant increases in error on some datasets.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n \u5f53Bagging\u548cBoosting\u8fdb\u884c\u76f4\u63a5\u6bd4\u8f83\u65f6\uff0cBoosting\u572819\u4e2a\u6570\u636e\u96c6\u4e0a\u9519\u8bef\u7387\u7684\u964d\u4f4e\u5e45\u5ea6\u66f4\u5927\uff0c\u4f18\u4e8eBagging\uff0c\u4e14\u57280.01\u7684\u663e\u8457\u6027\u6c34\u5e73\u4e0a\u663e\u8457\u3002\u7136\u800c\uff0cBoosting\u7684\u6548\u679c\u66f4\u52a0\u4e0d\u7a33\u5b9a\uff0c\u5728\u67d0\u4e9b\u6570\u636e\u96c6\u4e0a\u663e\u8457\u589e\u52a0\u4e86\u9519\u8bef\u7387\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u8fd9\u6bb5\u8bdd\u5bf9Bagging\u548cBoosting\u7684\u76f4\u63a5\u6bd4\u8f83\u8fdb\u884c\u4e86\u603b\u7ed3\uff0c\u5f3a\u8c03\u4e86Boosting\u5728\u9519\u8bef\u7387\u51cf\u5c11\u65b9\u9762\u7684\u4f18\u52bf\uff0c\u4f46\u540c\u65f6\u4e5f\u6307\u51fa\u4e86\u5176\u7ed3\u679c\u7684\u4e0d\u7a33\u5b9a\u6027\u3002\u8fd9\u79cd\u4e0d\u786e\u5b9a\u6027\u4f7f\u5f97Boosting\u5728\u5e94\u7528\u65f6\u9700\u8981\u66f4\u52a0\u8c28\u614e\uff0c\u5c24\u5176\u662f\u5728\u9762\u4e34\u4e0d\u5747\u8861\u6570\u636e\u6216\u7279\u6b8a\u6570\u636e\u96c6\u65f6\u3002<\/p>\n\n\n \u201cThe difference is highlighted in Figure 1, which compares bagging and boosting on two datasets: chess and colic. As T increases, the performance of bagging usually improves, but boosting can lead to a rapid degradation as in the colic dataset.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n \u56fe1\u7a81\u51fa\u4e86\u8fd9\u79cd\u5dee\u5f02\uff0c\u6bd4\u8f83\u4e86Bagging\u548cBoosting\u5728\u4e24\u4e2a\u6570\u636e\u96c6\uff08chess\u548ccolic\uff09\u4e0a\u7684\u8868\u73b0\u3002\u968f\u7740T\u7684\u589e\u52a0\uff0cBagging\u7684\u6027\u80fd\u901a\u5e38\u4f1a\u63d0\u9ad8\uff0c\u4f46Boosting\u5728colic\u6570\u636e\u96c6\u4e0a\u53ef\u80fd\u5bfc\u81f4\u6027\u80fd\u8fc5\u901f\u4e0b\u964d\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u901a\u8fc7\u56fe\u8868\u6bd4\u8f83\uff0c\u4f5c\u8005\u76f4\u89c2\u5730\u5c55\u793a\u4e86Bagging\u548cBoosting\u5728\u4e0d\u540c\u6570\u636e\u96c6\u4e0a\u7684\u8868\u73b0\u5dee\u5f02\uff0c\u5c24\u5176\u662fBoosting\u5728\u67d0\u4e9b\u6570\u636e\u96c6\u4e0a\u7684\u6027\u80fd\u6076\u5316\u3002\u8fd9\u8fdb\u4e00\u6b65\u8868\u660eBoosting\u5c3d\u7ba1\u5728\u6574\u4f53\u4e0a\u66f4\u6709\u6548\uff0c\u4f46\u5728\u67d0\u4e9b\u7279\u5b9a\u60c5\u51b5\u4e0b\u53ef\u80fd\u5b58\u5728\u8f83\u5927\u7684\u98ce\u9669\u3002<\/p>\n\n\n \u603b\u7ed3\u8bc4\u8bba<\/p>\n\n\n \u5b9e\u9a8c\u7ae0\u8282\u901a\u8fc7\u8be6\u5c3d\u7684\u5b9e\u9a8c\u8bbe\u8ba1\u548c\u7ed3\u679c\u5206\u6790\uff0c\u5c55\u793a\u4e86Bagging\u548cBoosting\u5728\u63d0\u9ad8\u5206\u7c7b\u5668\u51c6\u786e\u6027\u65b9\u9762\u7684\u4e0d\u540c\u8868\u73b0\u3002\u4f5c\u8005\u4e0d\u4ec5\u901a\u8fc7\u6570\u636e\u548c\u7edf\u8ba1\u7ed3\u679c\u8bc1\u660e\u4e86\u4e24\u79cd\u65b9\u6cd5\u7684\u6709\u6548\u6027\uff0c\u8fd8\u6df1\u5165\u63a2\u8ba8\u4e86Boosting\u53ef\u80fd\u5bfc\u81f4\u7684\u4e0d\u7a33\u5b9a\u6027\u548c\u98ce\u9669\u3002\u8fd9\u4e00\u7ae0\u8282\u4e3a\u8bfb\u8005\u63d0\u4f9b\u4e86\u5927\u91cf\u5b9e\u8bc1\u8bc1\u636e\uff0c\u4f7f\u4ed6\u4eec\u80fd\u591f\u66f4\u597d\u5730\u7406\u89e3\u548c\u5e94\u7528\u8fd9\u4e9b\u65b9\u6cd5\uff0c\u5e76\u5728\u5b9e\u8df5\u4e2d\u6743\u8861\u5b83\u4eec\u7684\u4f18\u52a3\u3002<\/p>\n\n\n \u201cFreund and Schapire (1996a) put this down to overfitting\u2014a large number of trials T allows the composite classifier C to become very complex.\u201d*<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n Freund\u548cSchapire\uff081996a\uff09\u5c06\u6b64\u5f52\u56e0\u4e8e\u8fc7\u62df\u5408\u2014\u2014\u5927\u91cf\u7684\u8bd5\u9a8c\u6b21\u6570T\u4f7f\u5f97\u7ec4\u5408\u5206\u7c7b\u5668C*\u53d8\u5f97\u975e\u5e38\u590d\u6742\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u8fd9\u6bb5\u8bdd\u6307\u51fa\u4e86Boosting\u5728\u67d0\u4e9b\u60c5\u51b5\u4e0b\u5931\u8d25\u7684\u4e00\u4e2a\u4e3b\u8981\u539f\u56e0\uff1a\u8fc7\u62df\u5408<\/strong>\u3002\u968f\u7740\u8bd5\u9a8c\u6b21\u6570\u7684\u589e\u52a0\uff0c\u6a21\u578b\u7684\u590d\u6742\u5ea6\u4e0a\u5347\uff0c\u5bfc\u81f4\u5b83\u5bf9\u8bad\u7ec3\u6570\u636e\u7684\u62df\u5408\u8fc7\u4e8e\u7cbe\u7ec6\uff0c\u4ece\u800c\u964d\u4f4e\u4e86\u5728\u65b0\u6570\u636e\u4e0a\u7684\u6cdb\u5316\u80fd\u529b\u3002\u8fd9\u4e00\u5206\u6790\u4e3a\u7406\u89e3Boosting\u5728\u590d\u6742\u6570\u636e\u96c6\u4e0a\u8868\u73b0\u4e0d\u4f73\u63d0\u4f9b\u4e86\u91cd\u8981\u7684\u7406\u8bba\u4f9d\u636e\u3002<\/p>\n\n\n \u201cThe objective of boosting is to construct a classifier C that performs well on the training data even when its constituent classifiers Ct are weak.\u201d*<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n Boosting\u7684\u76ee\u6807\u662f\u5373\u4f7f\u5176\u7ec4\u6210\u5206\u7c7b\u5668Ct\u8f83\u5f31\uff0c\u4e5f\u80fd\u6784\u5efa\u4e00\u4e2a\u5728\u8bad\u7ec3\u6570\u636e\u4e0a\u8868\u73b0\u826f\u597d\u7684\u5206\u7c7b\u5668C*\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u8fd9\u6bb5\u8bdd\u5f3a\u8c03\u4e86Boosting\u7684\u8bbe\u8ba1\u521d\u8877\uff0c\u5373\u901a\u8fc7\u7ec4\u5408\u591a\u4e2a\u5f31\u5206\u7c7b\u5668\u6765\u5f62\u6210\u4e00\u4e2a\u5f3a\u5206\u7c7b\u5668\u3002\u7136\u800c\uff0c\u8fd9\u4e5f\u53ef\u80fd\u5bfc\u81f4\u5728\u67d0\u4e9b\u60c5\u51b5\u4e0b\u6a21\u578b\u8fc7\u5ea6\u5173\u6ce8\u8bad\u7ec3\u6570\u636e\uff0c\u5ffd\u7565\u4e86\u6cdb\u5316\u6027\u80fd\uff0c\u8fd9\u6b63\u662fBoosting\u6709\u65f6\u4f1a\u5931\u8d25\u7684\u539f\u56e0\u4e4b\u4e00\u3002<\/p>\n\n\n \u201cFurther trials in this situation would seem to offer no gain\u2014they will increase the complexity of C but cannot improve its performance on the training data.\u201d*<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n \u5728\u8fd9\u79cd\u60c5\u51b5\u4e0b\uff0c\u8fdb\u4e00\u6b65\u7684\u8bd5\u9a8c\u4f3c\u4e4e\u4e0d\u4f1a\u5e26\u6765\u4efb\u4f55\u6536\u76ca\u2014\u2014\u5b83\u4eec\u53ea\u4f1a\u589e\u52a0C*\u7684\u590d\u6742\u6027\uff0c\u4f46\u65e0\u6cd5\u63d0\u9ad8\u5176\u5728\u8bad\u7ec3\u6570\u636e\u4e0a\u7684\u8868\u73b0\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u4f5c\u8005\u5728\u8fd9\u91cc\u6307\u51fa\uff0c\u5f53Boosting\u7684\u7ec4\u6210\u5206\u7c7b\u5668\u5df2\u7ecf\u80fd\u591f\u5f88\u597d\u5730\u62df\u5408\u8bad\u7ec3\u6570\u636e\u65f6\uff0c\u7ee7\u7eed\u589e\u52a0\u8bd5\u9a8c\u6b21\u6570\u4e0d\u4ec5\u6ca1\u6709\u5e2e\u52a9\uff0c\u53cd\u800c\u4f1a\u5bfc\u81f4\u6a21\u578b\u53d8\u5f97\u8fc7\u4e8e\u590d\u6742\u3002\u8fd9\u4e00\u89c2\u5bdf\u4e3a\u4f18\u5316Boosting\u7b97\u6cd5\u63d0\u4f9b\u4e86\u601d\u8def\uff0c\u5373\u5728\u6a21\u578b\u590d\u6742\u5ea6\u548c\u6027\u80fd\u4e4b\u95f4\u627e\u5230\u5e73\u8861\u3002<\/p>\n\n\n \u201cDespite using fewer trials, and thus being less prone to overfitting, C4.5\u2019s generalization performance is worse.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n \u5c3d\u7ba1\u4f7f\u7528\u4e86\u66f4\u5c11\u7684\u8bd5\u9a8c\u6b21\u6570\uff0c\u4ece\u800c\u51cf\u5c11\u4e86\u8fc7\u62df\u5408\u7684\u503e\u5411\uff0c\u4f46C4.5\u7684\u6cdb\u5316\u6027\u80fd\u5374\u66f4\u5dee\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u8fd9\u6bb5\u8bdd\u63ed\u793a\u4e86\u4e00\u4e2a\u6709\u8da3\u7684\u73b0\u8c61\uff0c\u5373\u51cf\u5c11\u8bd5\u9a8c\u6b21\u6570\u867d\u7136\u80fd\u591f\u964d\u4f4e\u8fc7\u62df\u5408\u7684\u98ce\u9669\uff0c\u4f46\u5374\u53ef\u80fd\u65e0\u6cd5\u6539\u5584\u751a\u81f3\u4f1a\u6076\u5316\u6a21\u578b\u7684\u6cdb\u5316\u6027\u80fd\u3002\u8fd9\u8bf4\u660eBoosting\u7684\u6548\u679c\u5e76\u4e0d\u4ec5\u4ec5\u4f9d\u8d56\u4e8e\u8bd5\u9a8c\u6b21\u6570\uff0c\u8fd8\u6d89\u53ca\u5230\u5982\u4f55\u6709\u6548\u5730\u5229\u7528\u8bd5\u9a8c\u7ed3\u679c\u6765\u63d0\u5347\u6574\u4f53\u6a21\u578b\u7684\u6027\u80fd\u3002<\/strong><\/p>\n\n\n \u201cIt also calls into question the hypothesis that overfitting is sufficient to explain boosting\u2019s failure on some datasets, since much of the benefit realized by boosting seems to be caused by overfitting.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n \u8fd9\u4e5f\u8d28\u7591\u4e86\u8fc7\u62df\u5408\u8db3\u4ee5\u89e3\u91caBoosting\u5728\u67d0\u4e9b\u6570\u636e\u96c6\u4e0a\u5931\u8d25\u7684\u5047\u8bbe\uff0c\u56e0\u4e3aBoosting\u5e26\u6765\u7684\u8bb8\u591a\u597d\u5904\u4f3c\u4e4e\u6b63\u662f\u7531\u8fc7\u62df\u5408\u5f15\u8d77\u7684\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u4f5c\u8005\u5728\u8fd9\u91cc\u63d0\u51fa\u4e86\u4e00\u4e2a\u77db\u76fe\u7684\u89c2\u70b9\uff1a\u5c3d\u7ba1\u8fc7\u62df\u5408\u5e38\u5e38\u88ab\u8ba4\u4e3a\u662fBoosting\u5931\u8d25\u7684\u539f\u56e0\uff0c\u4f46\u5b9e\u9645\u4e0a\uff0c\u8fc7\u62df\u5408\u5728\u67d0\u79cd\u7a0b\u5ea6\u4e0a\u4e5f\u662fBoosting\u6210\u529f\u7684\u5173\u952e\u3002<\/strong>\u8fd9\u4e00\u89c2\u70b9\u8868\u660e\uff0cBoosting\u7684\u673a\u5236\u975e\u5e38\u590d\u6742\uff0c\u9700\u8981\u6df1\u5165\u7814\u7a76\u5176\u5728\u4e0d\u540c\u6570\u636e\u96c6\u4e0a\u7684\u8868\u73b0\uff0c\u4ee5\u7406\u89e3\u5176\u6210\u529f\u4e0e\u5931\u8d25\u7684\u771f\u6b63\u539f\u56e0\u3002<\/p>\n\n\n \u603b\u7ed3\u8bc4\u8bba<\/p>\n\n\n \u201cBoosting\u5931\u8d25\u7684\u539f\u56e0\u5206\u6790\u201d\u7ae0\u8282\u6df1\u5165\u63a2\u8ba8\u4e86Boosting\u65b9\u6cd5\u5728\u67d0\u4e9b\u60c5\u51b5\u4e0b\u5931\u6548\u7684\u53ef\u80fd\u539f\u56e0\uff0c\u7279\u522b\u662f\u8fc7\u62df\u5408\u7684\u95ee\u9898\u3002\u901a\u8fc7\u5206\u6790\u548c\u8d28\u7591\u4f20\u7edf\u89c2\u70b9\uff0c\u4f5c\u8005\u63ed\u793a\u4e86Boosting\u5728\u590d\u6742\u6570\u636e\u96c6\u4e0a\u8868\u73b0\u4e0d\u4f73\u7684\u6f5c\u5728\u539f\u56e0\uff0c\u5e76\u5f3a\u8c03\u4e86\u5728\u5b9e\u9645\u5e94\u7528\u4e2d\u5bf9\u6a21\u578b\u590d\u6742\u5ea6\u8fdb\u884c\u5408\u7406\u63a7\u5236\u7684\u91cd\u8981\u6027\u3002\u8fd9\u4e00\u7ae0\u8282\u5bf9Boosting\u65b9\u6cd5\u7684\u7406\u89e3\u63d0\u4f9b\u4e86\u66f4\u52a0\u5168\u9762\u7684\u89c6\u89d2\uff0c\u5e76\u4e3a\u672a\u6765\u7684\u7814\u7a76\u65b9\u5411\u63d0\u51fa\u4e86\u6311\u6218\u3002<\/p>\n\n\n \u201cFreund and Schapire (1996a) explicitly consider the use by AdaBoost.M1 of confidence estimates provided by some learning systems. When instance x is classified by Ct, let Ht(x) be a number between 0 and 1 that represents some informal measure of the reliability of the prediction Ct(x).\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n Freund\u548cSchapire\uff081996a\uff09\u660e\u786e\u8003\u8651\u4e86\u5728AdaBoost.M1\u4e2d\u4f7f\u7528\u7531\u67d0\u4e9b\u5b66\u4e60\u7cfb\u7edf\u63d0\u4f9b\u7684\u7f6e\u4fe1\u5ea6\u4f30\u8ba1\u3002\u5f53\u5b9e\u4f8bx\u88ab\u5206\u7c7b\u5668Ct\u5206\u7c7b\u65f6\uff0cHt(x)\u662f\u4e00\u4e2a\u4ecb\u4e8e0\u548c1\u4e4b\u95f4\u7684\u6570\u503c\uff0c\u8868\u793a\u9884\u6d4bCt(x)\u7684\u53ef\u9760\u6027\u7684\u4e00\u79cd\u975e\u6b63\u5f0f\u5ea6\u91cf\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u8fd9\u6bb5\u8bdd\u4ecb\u7ecd\u4e86Boosting\u65b9\u6cd5\u4e2d\u7684\u7f6e\u4fe1\u5ea6\u4f30\u8ba1\u6982\u5ff5\uff0c\u8fd9\u662f\u5bf9\u539f\u6709\u6295\u7968\u673a\u5236\u7684\u4e00\u79cd\u6539\u8fdb\u3002\u901a\u8fc7\u5f15\u5165\u7f6e\u4fe1\u5ea6\uff0cBoosting\u4e0d\u4ec5\u8003\u8651\u4e86\u5206\u7c7b\u5668\u7684\u51c6\u786e\u6027\uff0c\u8fd8\u5f15\u5165\u4e86\u5206\u7c7b\u5668\u5bf9\u5176\u9884\u6d4b\u7684\u4fe1\u5fc3\u7a0b\u5ea6\uff0c\u8fd9\u53ef\u4ee5\u4f7f\u6295\u7968\u673a\u5236\u66f4\u52a0\u7075\u6d3b\u548c\u7cbe\u51c6\u3002<\/p>\n\n\n \u201cAn alternative use of the confidence estimate Ht is in combining the predictions of the classifiers Ct to give the final prediction C(x) of the class of instance x. Instead of using the fixed weight log(1\/\u03b5t) for the vote of classifier Ct, it seems plausible to allow the voting weight of Ct to vary in response to the confidence with which x is classified.\u201d*<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n \u53e6\u4e00\u79cd\u4f7f\u7528\u7f6e\u4fe1\u5ea6\u4f30\u8ba1Ht\u7684\u65b9\u6cd5\u662f\u5c06\u5176\u7528\u4e8e\u7ec4\u5408\u5206\u7c7b\u5668Ct\u7684\u9884\u6d4b\uff0c\u4ee5\u7ed9\u51fa\u5b9e\u4f8bx\u7684\u6700\u7ec8\u9884\u6d4bC*(x)\u3002\u4e0e\u5176\u4f7f\u7528\u56fa\u5b9a\u7684\u6743\u91cdlog(1\/\u03b5t)\u6765\u8ba1\u7b97\u5206\u7c7b\u5668Ct\u7684\u6295\u7968\u6743\u91cd\uff0c\u4f3c\u4e4e\u66f4\u5408\u7406\u7684\u662f\u8ba9Ct\u7684\u6295\u7968\u6743\u91cd\u6839\u636e\u5206\u7c7b\u5668\u5bf9x\u7684\u5206\u7c7b\u7f6e\u4fe1\u5ea6\u800c\u53d8\u5316\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u8fd9\u6bb5\u8bdd\u63d0\u51fa\u4e86\u5bf9Boosting\u6295\u7968\u673a\u5236\u7684\u6539\u8fdb\u5efa\u8bae\uff0c\u5373\u6839\u636e\u5206\u7c7b\u5668\u7684\u7f6e\u4fe1\u5ea6\u52a8\u6001\u8c03\u6574\u6295\u7968\u6743\u91cd\uff0c\u800c\u4e0d\u662f\u4f7f\u7528\u56fa\u5b9a\u7684\u6743\u91cd\u3002\u8fd9\u79cd\u6539\u8fdb\u53ef\u80fd\u4f7f\u5f97Boosting\u5728\u9762\u5bf9\u4e0d\u5747\u8861\u6216\u566a\u58f0\u6570\u636e\u65f6\u80fd\u591f\u66f4\u597d\u5730\u5e73\u8861\u5404\u4e2a\u5206\u7c7b\u5668\u7684\u8d21\u732e\uff0c\u4ece\u800c\u63d0\u5347\u6574\u4f53\u6a21\u578b\u7684\u6027\u80fd\u3002<\/p>\n\n\n \u201cResults show improvement on 21 of the 27 datasets, the same error rate on one dataset, and a higher error rate on only one of the 27 datasets (chess). Average error rate is approximately 5% less than that obtained with the original voting weights.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n \u7ed3\u679c\u663e\u793a\uff0c\u572827\u4e2a\u6570\u636e\u96c6\u4e2d\u768421\u4e2a\u4e0a\u6709\u6240\u6539\u8fdb\uff0c\u5728\u4e00\u4e2a\u6570\u636e\u96c6\u4e0a\u7684\u9519\u8bef\u7387\u4fdd\u6301\u4e0d\u53d8\uff0c\u800c\u572827\u4e2a\u6570\u636e\u96c6\u4e2d\u4ec5\u6709\u4e00\u4e2a\u6570\u636e\u96c6\uff08chess\uff09\u7684\u9519\u8bef\u7387\u6709\u6240\u4e0a\u5347\u3002\u5e73\u5747\u9519\u8bef\u7387\u6bd4\u4f7f\u7528\u539f\u59cb\u6295\u7968\u6743\u91cd\u65f6\u51cf\u5c11\u4e86\u5927\u7ea65%\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u8fd9\u6bb5\u8bdd\u603b\u7ed3\u4e86\u8c03\u6574\u6295\u7968\u6743\u91cd\u540e\u7684\u5b9e\u9a8c\u7ed3\u679c\uff0c\u8868\u660e\u8fd9\u79cd\u6539\u8fdb\u5728\u5927\u591a\u6570\u60c5\u51b5\u4e0b\u90fd\u80fd\u6709\u6548\u63d0\u9ad8\u5206\u7c7b\u5668\u7684\u6027\u80fd\uff0c\u7279\u522b\u662f\u5e73\u5747\u9519\u8bef\u7387\u7684\u663e\u8457\u964d\u4f4e\uff0c\u9a8c\u8bc1\u4e86\u52a8\u6001\u6743\u91cd\u673a\u5236\u7684\u6709\u6548\u6027\u3002\u7136\u800c\uff0c\u5728\u5c11\u6570\u60c5\u51b5\u4e0b\uff0c\u6539\u8fdb\u540e\u7684\u65b9\u6cd5\u53ef\u80fd\u5bfc\u81f4\u6027\u80fd\u4e0b\u964d\uff0c\u8fd9\u63d0\u793a\u6211\u4eec\u5728\u4f7f\u7528\u8fd9\u79cd\u65b9\u6cd5\u65f6\u4ecd\u9700\u8c28\u614e\u3002<\/p>\n\n\n \u201cThis modification is necessarily ad-hoc, since the confidence estimate Ht has only an intuitive meaning. However, it will be interesting to experiment with other voting schemes, and to see whether any of them can be used to give error bounds similar to those proved for the original boosting method.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n \u8fd9\u79cd\u4fee\u6539\u5fc5\u7136\u662f\u4e34\u65f6\u6027\u7684\uff0c\u56e0\u4e3a\u7f6e\u4fe1\u5ea6\u4f30\u8ba1Ht\u4ec5\u5177\u6709\u76f4\u89c2\u7684\u610f\u4e49\u3002\u7136\u800c\uff0c\u5c1d\u8bd5\u5176\u4ed6\u6295\u7968\u65b9\u6848\u5e76\u89c2\u5bdf\u5b83\u4eec\u662f\u5426\u80fd\u63d0\u4f9b\u7c7b\u4f3c\u4e8e\u539f\u59cbBoosting\u65b9\u6cd5\u6240\u8bc1\u660e\u7684\u9519\u8bef\u8fb9\u754c\uff0c\u5c06\u4f1a\u662f\u4e00\u4e2a\u6709\u8da3\u7684\u5b9e\u9a8c\u65b9\u5411\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u4f5c\u8005\u5728\u8fd9\u91cc\u627f\u8ba4\u4e86\u4f7f\u7528\u7f6e\u4fe1\u5ea6\u8c03\u6574\u6295\u7968\u6743\u91cd\u7684\u65b9\u6cd5\u5177\u6709\u4e00\u5b9a\u7684\u5373\u5174\u6027\uff0c\u5f3a\u8c03\u4e86\u8fd9\u4e00\u65b9\u6cd5\u7684\u76f4\u89c2\u6027\u548c\u6f5c\u5728\u5c40\u9650\u6027\u3002\u540c\u65f6\uff0c\u4f5c\u8005\u4e5f\u63d0\u51fa\u4e86\u672a\u6765\u7814\u7a76\u7684\u65b9\u5411\uff0c\u5373\u63a2\u7d22\u5176\u4ed6\u6295\u7968\u673a\u5236\uff0c\u8bd5\u56fe\u627e\u5230\u65e2\u80fd\u63d0\u4f9b\u7c7b\u4f3c\u9519\u8bef\u8fb9\u754c\u53c8\u5177\u5907\u7406\u8bba\u652f\u6301\u7684\u65b9\u6cd5\u3002<\/p>\n\n\n \u603b\u7ed3\u8bc4\u8bba<\/p>\n\n\n \u201c\u8c03\u6574\u6295\u7968\u6743\u91cd\u201d\u7ae0\u8282\u4ecb\u7ecd\u4e86\u901a\u8fc7\u5f15\u5165\u7f6e\u4fe1\u5ea6\u6765\u6539\u8fdbBoosting\u6295\u7968\u673a\u5236\u7684\u65b9\u6cd5\uff0c\u5e76\u901a\u8fc7\u5b9e\u9a8c\u9a8c\u8bc1\u4e86\u8fd9\u79cd\u65b9\u6cd5\u5728\u5927\u591a\u6570\u6570\u636e\u96c6\u4e0a\u5e26\u6765\u7684\u6027\u80fd\u63d0\u5347\u3002\u5c3d\u7ba1\u8fd9\u4e00\u6539\u8fdb\u5177\u6709\u4e00\u5b9a\u7684\u5373\u5174\u6027\uff0c\u7f3a\u4e4f\u4e25\u5bc6\u7684\u7406\u8bba\u652f\u6301\uff0c\u4f46\u5b83\u4e3a\u63d0\u9ad8Boosting\u7684\u7075\u6d3b\u6027\u548c\u9002\u5e94\u6027\u63d0\u4f9b\u4e86\u65b0\u7684\u601d\u8def\u3002\u4f5c\u8005\u4e5f\u8ba4\u8bc6\u5230\u8fd9\u4e00\u6539\u8fdb\u7684\u5c40\u9650\u6027\uff0c\u5e76\u4e3a\u672a\u6765\u7684\u7814\u7a76\u63d0\u51fa\u4e86\u63a2\u7d22\u65b0\u6295\u7968\u673a\u5236\u7684\u5efa\u8bae\uff0c\u8fd9\u4e3a\u8fdb\u4e00\u6b65\u4f18\u5316Boosting\u7b97\u6cd5\u63d0\u4f9b\u4e86\u65b9\u5411\u3002<\/p>\n\n\n \u201cTrials over a diverse collection of datasets have confirmed that boosted and bagged versions of C4.5 produce noticeably more accurate classifiers than the standard version.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n \u5728\u5404\u79cd\u6570\u636e\u96c6\u4e0a\u7684\u5b9e\u9a8c\u5df2\u7ecf\u8bc1\u5b9e\uff0c\u7ecf\u8fc7Boosting\u548cBagging\u5904\u7406\u7684C4.5\u7248\u672c\u80fd\u591f\u751f\u6210\u660e\u663e\u6bd4\u6807\u51c6\u7248\u672c\u66f4\u4e3a\u51c6\u786e\u7684\u5206\u7c7b\u5668\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u8fd9\u6bb5\u8bdd\u603b\u7ed3\u4e86\u5b9e\u9a8c\u7684\u4e3b\u8981\u53d1\u73b0\uff0c\u5f3a\u8c03\u4e86Boosting\u548cBagging\u65b9\u6cd5\u5728\u63d0\u9ad8C4.5\u5206\u7c7b\u5668\u6027\u80fd\u65b9\u9762\u7684\u6709\u6548\u6027\u3002\u8fd9\u79cd\u7ed3\u8bba\u5bf9\u673a\u5668\u5b66\u4e60\u4ece\u4e1a\u8005\u6765\u8bf4\u975e\u5e38\u91cd\u8981\uff0c\u56e0\u4e3a\u5b83\u9a8c\u8bc1\u4e86\u8fd9\u4e9b\u6280\u672f\u5728\u5b9e\u9645\u5e94\u7528\u4e2d\u7684\u53ef\u884c\u6027\u548c\u4f18\u8d8a\u6027\u3002<\/p>\n\n\n \u201cBoosting seems to be more effective than bagging when applied to C4.5, although the performance of the bagged C4.5 is less variable than its boosted counterpart.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n Boosting\u5728\u5e94\u7528\u4e8eC4.5\u65f6\u4f3c\u4e4e\u6bd4Bagging\u66f4\u6709\u6548\uff0c\u5c3d\u7ba1Bagging\u5904\u7406\u8fc7\u7684C4.5\u6027\u80fd\u53d8\u5316\u8f83\u5c0f\uff0c\u6bd4Boosting\u66f4\u7a33\u5b9a\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u8fd9\u6bb5\u8bdd\u5bf9Boosting\u548cBagging\u8fdb\u884c\u4e86\u76f4\u63a5\u5bf9\u6bd4\uff0c\u6307\u51faBoosting\u867d\u7136\u5728\u591a\u6570\u60c5\u51b5\u4e0b\u8868\u73b0\u66f4\u597d\uff0c\u4f46\u5176\u6027\u80fd\u6ce2\u52a8\u8f83\u5927\uff0c\u8fd9\u610f\u5473\u7740\u5728\u67d0\u4e9b\u5e94\u7528\u573a\u666f\u4e0b\u9700\u8981\u6743\u8861\u9009\u62e9\u3002\u8fd9\u4e3a\u8bfb\u8005\u5728\u9009\u62e9\u548c\u5e94\u7528\u8fd9\u4e9b\u6280\u672f\u65f6\u63d0\u4f9b\u4e86\u6709\u4ef7\u503c\u7684\u53c2\u8003\u3002<\/p>\n\n\n \u201cIf the voting weights used to aggregate component classifiers into a boosted classifier are altered to reflect the confidence with which individual instances are classified, better results are obtained on almost all the datasets investigated.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n \u5982\u679c\u8c03\u6574\u7528\u4e8e\u5c06\u5404\u4e2a\u7ec4\u6210\u5206\u7c7b\u5668\u805a\u5408\u6210Boosting\u5206\u7c7b\u5668\u7684\u6295\u7968\u6743\u91cd\uff0c\u4ee5\u53cd\u6620\u4e2a\u522b\u5b9e\u4f8b\u7684\u5206\u7c7b\u7f6e\u4fe1\u5ea6\uff0c\u90a3\u4e48\u51e0\u4e4e\u5728\u6240\u6709\u88ab\u8c03\u67e5\u7684\u6570\u636e\u96c6\u4e0a\u90fd\u53ef\u4ee5\u83b7\u5f97\u66f4\u597d\u7684\u7ed3\u679c\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u8fd9\u6bb5\u8bdd\u518d\u6b21\u5f3a\u8c03\u4e86\u5728Boosting\u4e2d\u8c03\u6574\u6295\u7968\u6743\u91cd\u7684\u91cd\u8981\u6027\uff0c\u8868\u660e\u901a\u8fc7\u5f15\u5165\u7f6e\u4fe1\u5ea6\u53ef\u4ee5\u663e\u8457\u63d0\u5347\u6a21\u578b\u7684\u6027\u80fd\u3002\u8fd9\u4e3a\u8fdb\u4e00\u6b65\u4f18\u5316Boosting\u7b97\u6cd5\u63d0\u4f9b\u4e86\u542f\u793a\uff0c\u8868\u660e\u5bf9\u6295\u7968\u673a\u5236\u8fdb\u884c\u7ec6\u5fae\u8c03\u6574\u53ef\u80fd\u4f1a\u5e26\u6765\u663e\u8457\u7684\u6539\u8fdb\u3002<\/p>\n\n\n \u201cA better understanding of why boosting sometimes fails is a clear desideratum at this point.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n \u66f4\u597d\u5730\u7406\u89e3\u4e3a\u4ec0\u4e48Boosting\u6709\u65f6\u4f1a\u5931\u8d25\u662f\u76ee\u524d\u7684\u4e00\u4e2a\u660e\u786e\u9700\u6c42\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u8fd9\u6bb5\u8bdd\u63d0\u51fa\u4e86\u5f53\u524d\u7814\u7a76\u4e2d\u7684\u4e00\u4e2a\u91cd\u8981\u6311\u6218\uff0c\u5373\u6df1\u5165\u63a2\u8ba8\u548c\u7406\u89e3Boosting\u5728\u67d0\u4e9b\u60c5\u51b5\u4e0b\u5931\u6548\u7684\u539f\u56e0\u3002\u8fd9\u4e00\u95ee\u9898\u7684\u89e3\u51b3\u5bf9\u4e8e\u8fdb\u4e00\u6b65\u63d0\u9ad8Boosting\u7b97\u6cd5\u7684\u9c81\u68d2\u6027\u548c\u666e\u9002\u6027\u81f3\u5173\u91cd\u8981\uff0c\u4e5f\u4e3a\u672a\u6765\u7684\u7814\u7a76\u6307\u660e\u4e86\u65b9\u5411\u3002<\/p>\n\n\n \u201cIt may be possible to modify the boosting approach and its associated proofs so that weights are adjusted separately within each class without changing overall class weights.\u201d<\/p>\n\n\n \u7ffb\u8bd1\uff1a<\/p>\n\n\n \u6709\u53ef\u80fd\u5bf9Boosting\u65b9\u6cd5\u53ca\u5176\u76f8\u5173\u8bc1\u660e\u8fdb\u884c\u4fee\u6539\uff0c\u4ee5\u4fbf\u5728\u4e0d\u6539\u53d8\u6574\u4f53\u7c7b\u522b\u6743\u91cd\u7684\u60c5\u51b5\u4e0b\uff0c\u5206\u522b\u5728\u6bcf\u4e2a\u7c7b\u522b\u5185\u8c03\u6574\u6743\u91cd\u3002<\/p>\n\n\n \u8bc4\u8bba\uff1a<\/p>\n\n\n \u4f5c\u8005\u5728\u8fd9\u91cc\u63d0\u51fa\u4e86\u4e00\u4e2a\u6539\u8fdbBoosting\u65b9\u6cd5\u7684\u6f5c\u5728\u65b9\u5411\uff0c\u5373\u5728\u7c7b\u522b\u5185\u90e8\u8fdb\u884c\u66f4\u7cbe\u7ec6\u7684\u6743\u91cd\u8c03\u6574\u3002\u8fd9\u79cd\u8c03\u6574\u6709\u53ef\u80fd\u8fdb\u4e00\u6b65\u63d0\u9ad8Boosting\u5728\u5904\u7406\u4e0d\u5e73\u8861\u6570\u636e\u96c6\u65f6\u7684\u6027\u80fd\uff0c\u7279\u522b\u662f\u5728\u590d\u6742\u7684\u591a\u7c7b\u95ee\u9898\u4e2d\uff0c\u8fd9\u5c06\u662f\u4e00\u4e2a\u503c\u5f97\u63a2\u7d22\u7684\u7814\u7a76\u65b9\u5411\u3002<\/p>\n\n\n \u603b\u7ed3\u8bc4\u8bba<\/p>\n\n\n \u7ed3\u8bba\u7ae0\u8282\u603b\u7ed3\u4e86\u672c\u6587\u7684\u4e3b\u8981\u53d1\u73b0\uff0c\u5f3a\u8c03\u4e86Boosting\u548cBagging\u5728\u63d0\u9ad8C4.5\u5206\u7c7b\u5668\u6027\u80fd\u65b9\u9762\u7684\u6709\u6548\u6027\u3002\u867d\u7136Boosting\u603b\u4f53\u4e0a\u66f4\u4e3a\u6709\u6548\uff0c\u4f46\u5176\u6027\u80fd\u7684\u6ce2\u52a8\u6027\u548c\u4e0d\u7a33\u5b9a\u6027\u4e5f\u88ab\u6307\u51fa\u3002\u4f5c\u8005\u63d0\u51fa\u4e86\u901a\u8fc7\u8c03\u6574\u6295\u7968\u6743\u91cd\u6765\u8fdb\u4e00\u6b65\u4f18\u5316Boosting\u7684\u65b9\u6cd5\uff0c\u5e76\u5f3a\u8c03\u4e86\u5bf9Boosting\u5931\u8d25\u539f\u56e0\u7684\u6df1\u5165\u7814\u7a76\u9700\u6c42\u3002\u6700\u540e\uff0c\u4f5c\u8005\u5efa\u8bae\u672a\u6765\u7684\u7814\u7a76\u53ef\u4ee5\u5c1d\u8bd5\u5728\u7c7b\u522b\u5185\u90e8\u8fdb\u884c\u6743\u91cd\u8c03\u6574\uff0c\u4ee5\u8fdb\u4e00\u6b65\u63d0\u5347\u7b97\u6cd5\u6027\u80fd\u3002\u8fd9\u4e9b\u603b\u7ed3\u548c\u5efa\u8bae\u4e3a\u540e\u7eed\u7814\u7a76\u63d0\u4f9b\u4e86\u91cd\u8981\u7684\u53c2\u8003\u65b9\u5411\uff0c\u5e76\u547c\u5401\u5b66\u672f\u754c\u5bf9Boosting\u7b97\u6cd5\u8fdb\u884c\u66f4\u6df1\u5165\u7684\u63a2\u7d22\u3002<\/p>\n\n\n a. \u57fa\u4e8e\u5f53\u524d\u6743\u91cd\u8bad\u7ec3\u5206\u7c7b\u5668<\/strong>:<\/p>\n\n\n b. \u8ba1\u7b97\u5206\u7c7b\u8bef\u5dee<\/strong>:<\/p>\n\n\n \\(\\displaystyle \\epsilon_t = \\sum_{i=1}^{N} w_t(i) \\cdot \\mathbb{I}(C_t(x_i) \\neq y_i)\\)<\/p>\n\n\n \u5176\u4e2d\uff0c\\(\\mathbb{I}(\\cdot)\\)\u662f\u6307\u793a\u51fd\u6570\uff0c\u5206\u7c7b\u9519\u8bef\u65f6\u4e3a1\uff0c\u6b63\u786e\u65f6\u4e3a0\u3002<\/p>\n\n\n c. \u8ba1\u7b97\u5206\u7c7b\u5668\u6743\u91cd<\/strong>:<\/p>\n\n\n \\(\\displaystyle \\alpha_t = \\frac{1}{2} \\ln \\left(\\frac{1-\\epsilon_t}{\\epsilon_t}\\right)\r\n\\)<\/p>\n\n\n \u5728\u6539\u8fdb\u7684\u7b97\u6cd5\u4e2d\uff0c\u4f5c\u8005\u5efa\u8bae\u6839\u636e\u5206\u7c7b\u5668\u7684\u7f6e\u4fe1\u5ea6\u6765\u8c03\u6574\\(\\alpha_t\\)\uff0c\u5373\u5c06\\(\\alpha_t\\)\u66ff\u6362\u4e3a\u4e0e\u5206\u7c7b\u7f6e\u4fe1\u5ea6\u76f8\u5173\u7684\u4e00\u4e2a\u52a8\u6001\u6743\u91cd\\(H_t(x)\\)\u3002<\/p>\n\n\n d. \u66f4\u65b0\u6837\u672c\u6743\u91cd<\/strong>:<\/p>\n\n\n \\(\\displaystyle w_{t+1}(i) = w_t(i) \\cdot \\exp(-\\alpha_t \\cdot y_i \\cdot C_t(x_i))\\)<\/p>\n\n\n \u7136\u540e\u8fdb\u884c\u5f52\u4e00\u5316\u5904\u7406\uff0c\u4f7f\u5f97\u6240\u6709\u6837\u672c\u6743\u91cd\u4e4b\u548c\u4e3a1\uff1a<\/p>\n\n\n \\(\\displaystyle w_{t+1}(i) = \\frac{w_{t+1}(i)}{\\sum_{j=1}^{N} w_{t+1}(j)}\\)<\/p>\n\n\n e. \u7f6e\u4fe1\u5ea6\u8c03\u6574<\/strong>:<\/p>\n\n\n \\(\\displaystyle C^*(x) = \\text{sign} \\left(\\sum_{t=1}^{T} \\alpha_t \\cdot C_t(x)\\right)\r\n\\)<\/p>\n\n\n \u5176\u4e2d\\(\\alpha_t\\)\u662f\u6839\u636e\u7f6e\u4fe1\u5ea6\u8c03\u6574\u540e\u7684\u5206\u7c7b\u5668\u6743\u91cd\u3002<\/p>\n\n\n \u6539\u8fdb\u540e\u7684Boosting\u7b97\u6cd5\u901a\u8fc7\u5f15\u5165\u7f6e\u4fe1\u5ea6\u6765\u8c03\u6574\u5206\u7c7b\u5668\u7684\u6295\u7968\u6743\u91cd\uff0c\u65e8\u5728\u589e\u5f3a\u7b97\u6cd5\u7684\u7075\u6d3b\u6027\u548c\u5bf9\u590d\u6742\u6570\u636e\u7684\u9002\u5e94\u6027\u3002\u5b9e\u9a8c\u7ed3\u679c\u8868\u660e\uff0c\u8fd9\u79cd\u8c03\u6574\u5728\u5927\u591a\u6570\u6570\u636e\u96c6\u4e0a\u5e26\u6765\u4e86\u66f4\u597d\u7684\u6027\u80fd\uff0c\u5c24\u5176\u662f\u964d\u4f4e\u4e86\u5206\u7c7b\u9519\u8bef\u7387\u3002<\/p>\n\n\n \u5728\u8fd9\u7bc7\u8bba\u6587\u4e2d\uff0c\u4f5c\u8005\u63d0\u51fa\u7684\u52a8\u6001\u6743\u91cd \\(H_t(x)\\) \u662f\u7528\u4e8e\u6539\u8fdb\u4f20\u7edf AdaBoost \u7b97\u6cd5\u4e2d\u7684\u56fa\u5b9a\u6295\u7968\u6743\u91cd \\(\\alpha_t\\)\u3002\u5177\u4f53\u6765\u8bf4\uff0c\\(H_t(x)\\) \u8868\u793a\u5206\u7c7b\u5668 \\(C_t\\) \u5bf9\u5b9e\u4f8b \\(x\\) \u5206\u7c7b\u7684\u7f6e\u4fe1\u5ea6\uff0c\u6570\u503c\u4ecb\u4e8e 0 \u548c 1 \u4e4b\u95f4\uff0c\u53cd\u6620\u4e86\u5206\u7c7b\u5668\u5728\u9884\u6d4b\u5b9e\u4f8b \\(x\\) \u65f6\u7684\u81ea\u4fe1\u7a0b\u5ea6\u3002<\/p>\n\n\n \u5728\u51b3\u7b56\u6811\u5206\u7c7b\u5668\uff08\u5982 C4.5\uff09\u7684\u60c5\u51b5\u4e0b\uff0c\u6bcf\u4e2a\u53f6\u8282\u70b9\u4f1a\u5305\u542b\u591a\u4e2a\u8bad\u7ec3\u5b9e\u4f8b\u3002\u5bf9\u4e8e\u7ed9\u5b9a\u7684\u5b9e\u4f8b \\(x\\)\uff0c\u5047\u8bbe\u5b83\u88ab\u5206\u7c7b\u5668 \\(C_t\\) \u5212\u5206\u5230\u4e00\u4e2a\u7279\u5b9a\u7684\u53f6\u8282\u70b9\uff0c\u8be5\u53f6\u8282\u70b9\u5305\u542b\u7684\u5b9e\u4f8b\u96c6\u5408\u4e3a \\(S\\)\u3002<\/p>\n\n\n \\(S\\) \u4e2d\u5c5e\u4e8e\u7c7b\u522b \\(k\\) \u7684\u5b9e\u4f8b\u6570\u91cf\u4e3a \\(|S_k|\\)\u3002<\/p>\n\n\n \u5bf9\u4e8e\u5b9e\u4f8b \\(x\\)\uff0c\u5047\u8bbe\u5206\u7c7b\u5668 \\(C_t\\) \u5c06\u5176\u5f52\u7c7b\u4e3a\u7c7b\u522b \\(k\\)\uff0c\u5219 \\(H_t(x)\\) \u53ef\u4ee5\u901a\u8fc7\u4ee5\u4e0b\u65b9\u5f0f\u4f30\u8ba1\uff1a<\/p>\n\n\n \\(\\displaystyle H_t(x) = \\frac{|S_k| + 1}{|S| + K}\\)<\/p>\n\n\n \u5176\u4e2d\uff0c\\(|S_k|\\) \u662f\u8282\u70b9 \\(S\\) \u4e2d\u5c5e\u4e8e\u7c7b\u522b \\(k\\) \u7684\u5b9e\u4f8b\u6570\u91cf\uff0c\\(|S|\\) \u662f\u8282\u70b9 \\(S\\) \u4e2d\u7684\u603b\u5b9e\u4f8b\u6570\u91cf\uff0c\\(K\\) \u662f\u7c7b\u522b\u7684\u603b\u6570\u3002<\/p>\n\n\n \u8fd9\u4e2a\u516c\u5f0f\u5229\u7528\u4e86\u62c9\u666e\u62c9\u65af\u5e73\u6ed1\uff08Laplace Smoothing\uff09\uff0c\u5373\u5728\u8ba1\u7b97\u6bd4\u4f8b\u65f6\u52a0\u4e0a 1\uff0c\u4ee5\u9632\u6b62\u7f6e\u4fe1\u5ea6\u4e3a 0 \u6216 1 \u7684\u6781\u7aef\u60c5\u51b5\u3002<\/p>\n\n\n \\(H_t(x)\\) \u7684\u503c\u8d8a\u63a5\u8fd1 1\uff0c\u8868\u793a\u5206\u7c7b\u5668\u5bf9\u5b9e\u4f8b \\(x\\) \u7684\u5206\u7c7b\u8d8a\u6709\u4fe1\u5fc3\u3002<\/p>\n\n\n \u76f8\u53cd\uff0c\\(H_t(x)\\) \u503c\u8d8a\u4f4e\uff0c\u8868\u793a\u5206\u7c7b\u5668\u5bf9\u5176\u5206\u7c7b\u7684\u4fe1\u5fc3\u8d8a\u5f31\u3002<\/p>\n\n\n \u5728\u6539\u8fdb\u540e\u7684 Boosting \u7b97\u6cd5\u4e2d\uff0c\\(H_t(x)\\) \u4e0d\u4ec5\u4ec5\u7528\u4e8e\u8861\u91cf\u5206\u7c7b\u5668\u5bf9\u5b9e\u4f8b\u7684\u5206\u7c7b\u4fe1\u5fc3\uff0c\u8fd8\u76f4\u63a5\u5f71\u54cd\u4e86\u8be5\u5206\u7c7b\u5668\u5728\u6700\u7ec8\u5206\u7c7b\u5668\u4e2d\u7684\u6295\u7968\u6743\u91cd\u3002\u5177\u4f53\u6765\u8bf4\uff0c\u539f\u59cb\u7684\u56fa\u5b9a\u6743\u91cd \\(\\alpha_t\\) \u5728\u6bcf\u4e00\u8f6e\u66f4\u65b0\u65f6\uff0c\u90fd\u4f1a\u88ab \\(H_t(x)\\) \u6240\u66ff\u4ee3\u6216\u4e0e\u4e4b\u7ed3\u5408\uff0c\u4ee5\u52a8\u6001\u8c03\u6574\u5206\u7c7b\u5668\u7684\u6295\u7968\u5f71\u54cd\u529b\u3002<\/p>\n\n\n \u5728\u6539\u8fdb\u540e\u7684 Boosting \u7b97\u6cd5\u4e2d\uff0c\\(H_t(x)\\) \u4f5c\u4e3a\u5206\u7c7b\u5668 \\(C_t\\) \u5bf9\u5b9e\u4f8b \\(x\\) \u5206\u7c7b\u7684\u7f6e\u4fe1\u5ea6\uff0c\u88ab\u7528\u4e8e\u8c03\u6574\u5206\u7c7b\u5668\u5728\u6700\u7ec8\u5206\u7c7b\u5668\u4e2d\u7684\u6295\u7968\u6743\u91cd\u3002\u5177\u4f53\u7684\u6b65\u9aa4\u5982\u4e0b\uff1a<\/p>\n\n\n \u5728\u4f20\u7edf\u7684 AdaBoost \u7b97\u6cd5\u4e2d\uff0c\u5206\u7c7b\u5668 \\(C_t\\) \u7684\u6295\u7968\u6743\u91cd \\(\\alpha_t\\) \u8ba1\u7b97\u516c\u5f0f\u4e3a\uff1a<\/p>\n\n\n \\(\\displaystyle \\alpha_t = \\frac{1}{2} \\ln \\left(\\frac{1 – \\epsilon_t}{\\epsilon_t}\\right)\\)<\/p>\n\n\n \u5176\u4e2d\uff0c\\(\\epsilon_t\\) \u662f\u5206\u7c7b\u5668 \\(C_t\\) \u7684\u9519\u8bef\u7387\u3002<\/p>\n\n\n \u5728\u6539\u8fdb\u7684\u7b97\u6cd5\u4e2d\uff0c\u6211\u4eec\u4f7f\u7528 \\(H_t(x)\\) \u6765\u8c03\u6574\u5206\u7c7b\u5668 \\(C_t\\) \u5bf9\u5b9e\u4f8b \\(x\\) \u7684\u6295\u7968\u6743\u91cd\u3002\u5177\u4f53\u505a\u6cd5\u662f\u5c06 \\(\\alpha_t\\) \u4e0e \\(H_t(x)\\) \u76f8\u7ed3\u5408\uff0c\u751f\u6210\u52a8\u6001\u8c03\u6574\u540e\u7684\u6295\u7968\u6743\u91cd\uff1a<\/p>\n\n\n \\(\\displaystyle \\alpha_t(x) = \\alpha_t \\cdot H_t(x)\\)<\/p>\n\n\n \u5176\u4e2d\uff0c\\(H_t(x)\\) \u662f\u5206\u7c7b\u5668 \\(C_t\\) \u5bf9\u5b9e\u4f8b \\(x\\) \u7684\u7f6e\u4fe1\u5ea6\uff0c\u503c\u5728 0 \u548c 1 \u4e4b\u95f4\u3002<\/p>\n\n\n \u6700\u7ec8\u5206\u7c7b\u5668 \\(C^*\\) \u662f\u6240\u6709\u5f31\u5206\u7c7b\u5668 \\(C_t\\) \u7684\u52a0\u6743\u7ec4\u5408\uff0c\u5bf9\u4e8e\u7ed9\u5b9a\u7684\u5b9e\u4f8b \\(x\\)\uff0c\u6700\u7ec8\u5206\u7c7b\u5668\u7684\u9884\u6d4b\u7ed3\u679c\u53ef\u4ee5\u8868\u793a\u4e3a\uff1a<\/p>\n\n\n \\(\\displaystyle C^*(x) = \\text{sign} \\left(\\sum_{t=1}^{T} \\alpha_t(x) \\cdot C_t(x)\\right)\\)<\/p>\n\n\n \u5176\u4e2d\uff0c\\(\\alpha_t(x)\\) \u662f\u6839\u636e\u7f6e\u4fe1\u5ea6 \\(H_t(x)\\) \u52a8\u6001\u8c03\u6574\u540e\u7684\u5206\u7c7b\u5668 \\(C_t\\) \u7684\u6295\u7968\u6743\u91cd\u3002<\/p>\n\n\n \u8fd9\u4e2a\u516c\u5f0f \\(C^*(x) = \\text{sign} \\left(\\sum_{t=1}^{T} \\alpha_t(x) \\cdot C_t(x)\\right)\\) \u63cf\u8ff0\u4e86\u6700\u7ec8\u5206\u7c7b\u5668 \\(C^*\\) \u662f\u5982\u4f55\u901a\u8fc7\u7ec4\u5408\u591a\u4e2a\u5f31\u5206\u7c7b\u5668 \\(C_t\\) \u6765\u5bf9\u4e00\u4e2a\u7ed9\u5b9a\u7684\u5b9e\u4f8b \\(x\\) \u8fdb\u884c\u5206\u7c7b\u7684\u3002\u4e0b\u9762\u662f\u5bf9\u8fd9\u4e2a\u516c\u5f0f\u7684\u8be6\u7ec6\u89e3\u91ca\uff1a<\/p>\n\n\n \u516c\u5f0f\u5404\u90e8\u5206\u7684\u542b\u4e49\uff1a<\/p>\n\n\n \u8fd9\u4e2a\u516c\u5f0f\u7684\u6838\u5fc3\u601d\u60f3\u662f\u901a\u8fc7\u5c06\u591a\u4e2a\u5f31\u5206\u7c7b\u5668\u7684\u9884\u6d4b\u7ed3\u679c\u8fdb\u884c\u52a0\u6743\u5e73\u5747\u6765\u63d0\u9ad8\u5206\u7c7b\u6027\u80fd\u3002\u6bcf\u4e2a\u5f31\u5206\u7c7b\u5668\u5728\u6700\u7ec8\u51b3\u7b56\u4e2d\u7684\u5f71\u54cd\u529b\u7531\u5176\u6295\u7968\u6743\u91cd \\(\\alpha_t(x)\\) \u51b3\u5b9a\uff0c\u800c\u6743\u91cd\u53c8\u6839\u636e\u5206\u7c7b\u5668\u5bf9\u8be5\u5b9e\u4f8b\u7684\u4fe1\u5fc3\uff08\u7f6e\u4fe1\u5ea6\uff09\u8fdb\u884c\u8c03\u6574\u3002\u8fd9\u610f\u5473\u7740\u5bf9\u90a3\u4e9b\u5206\u7c7b\u5668\u4fe1\u5fc3\u8f83\u9ad8\u7684\u9884\u6d4b\u7ed3\u679c\u4f1a\u6709\u66f4\u5927\u7684\u5f71\u54cd\u529b\uff0c\u53cd\u4e4b\u5219\u5f71\u54cd\u8f83\u5c0f\u3002<\/p>\n\n\n \u901a\u8fc7\u8fd9\u4e2a\u52a0\u6743\u6295\u7968\u673a\u5236\uff0c\u6700\u7ec8\u5206\u7c7b\u5668 \\(C^*(x)\\) \u80fd\u591f\u7efc\u5408\u591a\u4e2a\u5f31\u5206\u7c7b\u5668\u7684\u4fe1\u606f\uff0c\u5f97\u51fa\u66f4\u4e3a\u51c6\u786e\u548c\u9c81\u68d2\u7684\u5206\u7c7b\u7ed3\u679c\u3002\u8fd9\u4e5f\u662f Boosting \u6280\u672f\u80fd\u591f\u663e\u8457\u63d0\u5347\u5206\u7c7b\u6027\u80fd\u7684\u5173\u952e\u6240\u5728\u3002<\/p>\n\n\n \u901a\u8fc7\u5f15\u5165 \\(H_t(x)\\)\uff0c\u6211\u4eec\u4e3a\u6bcf\u4e2a\u5206\u7c7b\u5668\u7684\u6295\u7968\u6743\u91cd\u8d4b\u4e88\u4e86\u66f4\u591a\u7684\u7075\u6d3b\u6027\uff0c\u4f7f\u5f97\u5206\u7c7b\u5668\u5bf9\u90a3\u4e9b\u5b83\u66f4\u6709\u4fe1\u5fc3\u7684\u5b9e\u4f8b\u62e5\u6709\u66f4\u5927\u7684\u6295\u7968\u5f71\u54cd\u529b\u3002\u8fd9\u6837\u4e00\u6765\uff0c\u5206\u7c7b\u5668\u5728\u6700\u7ec8\u51b3\u7b56\u4e2d\u7684\u4f5c\u7528\u4e0d\u4ec5\u53d6\u51b3\u4e8e\u5176\u6574\u4f53\u9519\u8bef\u7387\uff0c\u8fd8\u53d6\u51b3\u4e8e\u5176\u5bf9\u6bcf\u4e2a\u5177\u4f53\u5b9e\u4f8b\u7684\u5206\u7c7b\u4fe1\u5fc3\uff0c\u4ece\u800c\u53ef\u80fd\u63d0\u5347\u6574\u4f53\u6a21\u578b\u7684\u6027\u80fd\uff0c\u7279\u522b\u662f\u5728\u5904\u7406\u590d\u6742\u6570\u636e\u65f6\u3002<\/p>\n","protected":false},"excerpt":{"rendered":" \u8bba\u6587\u7b80\u4ecb \u82f1\u6587\u9898\u76ee\uff1aBagging, Boosting, a…<\/p>\n","protected":false},"author":1,"featured_media":504,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[26],"tags":[27],"class_list":["post-345","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-26","tag-ensemble-learning"],"_links":{"self":[{"href":"https:\/\/www.xuzhe.tj.cn\/index.php\/wp-json\/wp\/v2\/posts\/345","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.xuzhe.tj.cn\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.xuzhe.tj.cn\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.xuzhe.tj.cn\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.xuzhe.tj.cn\/index.php\/wp-json\/wp\/v2\/comments?post=345"}],"version-history":[{"count":3,"href":"https:\/\/www.xuzhe.tj.cn\/index.php\/wp-json\/wp\/v2\/posts\/345\/revisions"}],"predecessor-version":[{"id":505,"href":"https:\/\/www.xuzhe.tj.cn\/index.php\/wp-json\/wp\/v2\/posts\/345\/revisions\/505"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.xuzhe.tj.cn\/index.php\/wp-json\/wp\/v2\/media\/504"}],"wp:attachment":[{"href":"https:\/\/www.xuzhe.tj.cn\/index.php\/wp-json\/wp\/v2\/media?parent=345"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.xuzhe.tj.cn\/index.php\/wp-json\/wp\/v2\/categories?post=345"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.xuzhe.tj.cn\/index.php\/wp-json\/wp\/v2\/tags?post=345"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}\u6587\u7ae0\u5404\u7ae0\u8282<\/h2>\n\n\n
\n
\u2022 \u7b80\u8981\u4ecb\u7ecd\u4e86Bagging\u548cBoosting\u8fd9\u4e24\u79cd\u65b9\u6cd5\u7684\u57fa\u672c\u539f\u7406\u53ca\u5176\u5728\u63d0\u9ad8\u5206\u7c7b\u5668\u9884\u6d4b\u7cbe\u5ea6\u65b9\u9762\u7684\u8868\u73b0\u3002
\u2022 \u63d0\u5230\u5b9e\u9a8c\u7ed3\u679c\uff0c\u8bf4\u660eBoosting\u603b\u4f53\u4e0a\u6548\u679c\u66f4\u597d\uff0c\u4f46\u5728\u67d0\u4e9b\u6570\u636e\u96c6\u4e0a\u53ef\u80fd\u4f1a\u5bfc\u81f4\u6027\u80fd\u4e0b\u964d\u3002
<\/li>\n
\u2022 \u4ecb\u7ecd\u4e86\u673a\u5668\u5b66\u4e60\u4e2d\u5bf9\u63d0\u9ad8\u9884\u6d4b\u7cbe\u5ea6\u7684\u7814\u7a76\u80cc\u666f\u3002
\u2022 \u63d0\u51fa\u4f7f\u7528\u591a\u4e2a\u5206\u7c7b\u5668\u7ec4\u5408\uff08\u5982Bagging\u548cBoosting\uff09\u6765\u63d0\u9ad8\u5206\u7c7b\u5668\u6027\u80fd\u7684\u57fa\u672c\u601d\u60f3\u3002
\u2022 \u63d0\u5230Bagging\u548cBoosting\u7684\u7406\u8bba\u57fa\u7840\u53ca\u5176\u9002\u7528\u8303\u56f4\uff0c\u7279\u522b\u662f\u5728\u51b3\u7b56\u6811\u5b66\u4e60\u4e2d\u7684\u5e94\u7528\u3002
<\/li>\n
\u2022 Bagging\uff1a \u8be6\u7ec6\u63cf\u8ff0\u4e86Bagging\u7684\u5de5\u4f5c\u539f\u7406\uff0c\u5305\u62ec\u5982\u4f55\u901a\u8fc7\u751f\u6210\u81ea\u52a9\u6837\u672c\u96c6\u5e76\u8fdb\u884c\u591a\u6b21\u8bad\u7ec3\u6765\u63d0\u9ad8\u5206\u7c7b\u5668\u7684\u7a33\u5b9a\u6027\u548c\u7cbe\u5ea6\u3002
\u2022 Boosting\uff1a \u8be6\u7ec6\u63cf\u8ff0\u4e86Boosting\u7684\u5de5\u4f5c\u673a\u5236\uff0c\u5c24\u5176\u662fAdaBoost\u7b97\u6cd5\u5982\u4f55\u901a\u8fc7\u8c03\u6574\u8bad\u7ec3\u6837\u672c\u7684\u6743\u91cd\u6765\u5173\u6ce8\u96be\u4ee5\u5206\u7c7b\u7684\u5b9e\u4f8b\uff0c\u8fdb\u800c\u63d0\u9ad8\u6574\u4f53\u5206\u7c7b\u5668\u7684\u51c6\u786e\u6027\u3002
\u2022 \u6bd4\u8f83\u4e86Bagging\u548cBoosting\u5728\u5904\u7406\u5206\u7c7b\u5668\u7ec4\u5408\u65f6\u7684\u4e0d\u540c\u7b56\u7565\uff0c\u5c24\u5176\u662f\u6295\u7968\u673a\u5236\u7684\u5dee\u5f02\u3002
<\/li>\n
\u2022 \u4ecb\u7ecd\u4e86\u5b9e\u9a8c\u8bbe\u8ba1\uff0c\u5305\u62ec\u4f7f\u7528\u7684\u6570\u636e\u96c6\uff08\u6765\u81eaUCI\u673a\u5668\u5b66\u4e60\u5e93\uff09\u3001\u5b9e\u9a8c\u65b9\u6cd5\uff0810\u6b2110\u6298\u4ea4\u53c9\u9a8c\u8bc1\uff09\u548c\u8bc4\u4ef7\u6307\u6807\u3002
\u2022 \u5b9e\u9a8c\u7ed3\u679c\uff1a \u5c55\u793a\u4e86Bagging\u548cBoosting\u5728\u4e0d\u540c\u6570\u636e\u96c6\u4e0a\u7684\u8868\u73b0\uff0c\u5217\u51fa\u4e86\u6bcf\u4e2a\u6570\u636e\u96c6\u7684\u5177\u4f53\u9519\u8bef\u7387\u4ee5\u53ca\u8fd9\u4e9b\u65b9\u6cd5\u5728\u4e0d\u540c\u8bd5\u9a8c\u6b21\u6570\u4e0b\u7684\u8868\u73b0\u3002
\u2022 \u901a\u8fc7\u6bd4\u8f83Bagging\u548cBoosting\u7684\u5b9e\u9a8c\u7ed3\u679c\uff0c\u5206\u6790\u4e86Boosting\u5728\u67d0\u4e9b\u60c5\u51b5\u4e0b\u6548\u679c\u4e0d\u4f73\u7684\u539f\u56e0\uff0c\u5982\u8fc7\u62df\u5408\u6216\u6743\u91cd\u5206\u5e03\u504f\u659c\u3002
<\/li>\n
\u2022 \u8fdb\u4e00\u6b65\u63a2\u8ba8\u4e86Boosting\u5728\u67d0\u4e9b\u6570\u636e\u96c6\u4e0a\u8868\u73b0\u4e0d\u4f73\u7684\u539f\u56e0\uff0c\u5c24\u5176\u662f\u5f53\u8bd5\u9a8c\u6b21\u6570\u8fc7\u591a\u65f6\uff0c\u53ef\u80fd\u5bfc\u81f4\u6a21\u578b\u8fc7\u62df\u5408\u3002
\u2022 \u63d0\u51fa\u4e86\u901a\u8fc7\u51cf\u5c11\u8bd5\u9a8c\u6b21\u6570\u548c\u8c03\u6574\u6295\u7968\u6743\u91cd\u6765\u7f13\u89e3\u8fd9\u4e00\u95ee\u9898\u7684\u6539\u8fdb\u65b9\u6cd5\uff0c\u5e76\u8fdb\u884c\u4e86\u5b9e\u9a8c\u9a8c\u8bc1\u3002
<\/li>\n
\u2022 \u4ecb\u7ecd\u4e86\u5bf9Boosting\u65b9\u6cd5\u7684\u8fdb\u4e00\u6b65\u6539\u8fdb\uff0c\u5373\u901a\u8fc7\u4f7f\u7528\u5206\u7c7b\u5668\u5bf9\u5b9e\u4f8b\u5206\u7c7b\u7684\u7f6e\u4fe1\u5ea6\u6765\u8c03\u6574\u6295\u7968\u6743\u91cd\uff0c\u4ece\u800c\u5728\u7ec4\u5408\u5206\u7c7b\u5668\u65f6\u66f4\u597d\u5730\u53cd\u6620\u6bcf\u4e2a\u5206\u7c7b\u5668\u7684\u53ef\u9760\u6027\u3002
\u2022 \u5b9e\u9a8c\u7ed3\u679c\u8868\u660e\uff0c\u7ecf\u8fc7\u8c03\u6574\u540e\u7684Boosting\u65b9\u6cd5\u5728\u5927\u591a\u6570\u6570\u636e\u96c6\u4e0a\u7684\u8868\u73b0\u4f18\u4e8e\u4f20\u7edf\u65b9\u6cd5\u3002
<\/li>\n
\u2022 \u603b\u7ed3\u4e86Bagging\u548cBoosting\u5728\u63d0\u9ad8\u5206\u7c7b\u5668\u7cbe\u5ea6\u65b9\u9762\u7684\u6709\u6548\u6027\u3002
\u2022 \u8ba8\u8bba\u4e86\u4e24\u79cd\u65b9\u6cd5\u7684\u4f18\u7f3a\u70b9\uff0c\u7279\u522b\u662fBoosting\u65b9\u6cd5\u5728\u67d0\u4e9b\u60c5\u51b5\u4e0b\u7684\u6f5c\u5728\u98ce\u9669\u3002
\u2022 \u63d0\u51fa\u4e86\u672a\u6765\u7684\u7814\u7a76\u65b9\u5411\uff0c\u5305\u62ec\u8fdb\u4e00\u6b65\u6539\u8fdbBoosting\u65b9\u6cd5\u7684\u6295\u7968\u673a\u5236\u3002
<\/li>\n
\u2022 \u63d0\u4f9b\u4e86\u5b9e\u9a8c\u4e2d\u4f7f\u7528\u7684\u6570\u636e\u96c6\u7684\u8be6\u7ec6\u4fe1\u606f\uff0c\u5305\u62ec\u6570\u636e\u96c6\u7684\u540d\u79f0\u3001\u6837\u672c\u6570\u91cf\u3001\u7c7b\u522b\u6570\u91cf\u3001\u5c5e\u6027\u7c7b\u578b\u7b49\u3002
<\/li>\n<\/ol>\n\n\n\u6458\u5f55<\/h2>\n\n\n
\u5f15\u8a00<\/h3>\n\n\n
\n
\n
\n
\n
Bagging and Boosting \u65b9\u6cd5 (Bagging and Boosting)
\n
\n
\n
\n
\n
\u5b9e\u9a8c (Experiments)<\/h3>\n\n\n
\n
\n
\n
\n
\n
Boosting\u5931\u8d25\u7684\u539f\u56e0\u5206\u6790 (Why Does Boosting Sometimes Fail)<\/h3>\n\n\n
\n
\n
\n
\n
\n
\u8c03\u6574\u6295\u7968\u6743\u91cd (Changing the Voting Weights)<\/h3>\n\n\n
\n
\n
\n
\n
\u7ed3\u8bba (Conclusion)<\/h3>\n\n\n
\n
\n
\n
\n
\n
\u6b64\u8bba\u6587\u4e2d\u63d0\u51fa\u7684 boosting \u6539\u8fdb\u7b97\u6cd5\u7684\u5177\u4f53\u7b97\u6cd5\u6b65\u9aa4<\/h2>\n\n\n
\u6539\u8fdb\u7684Boosting\u7b97\u6cd5\u6b65\u9aa4<\/h3>\n\n\n
\n
<\/li>\n
<\/li>\n<\/ol>\n\n\n\n
<\/li>\n<\/ul>\n\n\n\n
\n
\n
\n
<\/li>\n<\/ul>\n\n\n\n
\u5173\u952e\u6539\u8fdb\u70b9<\/h3>\n\n\n
\n
<\/li>\n<\/ul>\n\n\n\u7ed3\u8bba<\/h3>\n\n\n
\\(H_t(x)\\) \u7684\u8ba1\u7b97\u65b9\u6cd5<\/h2>\n\n\n
\u8ba1\u7b97\u52a8\u6001\u6743\u91cd \\(H_t(x)\\) \u7684\u6b65\u9aa4<\/h3>\n\n\n
\n
\n
<\/li>\n<\/ol>\n\n\n\n
<\/li>\n<\/ol>\n\n\n\u7f6e\u4fe1\u5ea6\u5728\u6295\u7968\u4e2d\u7684\u5e94\u7528<\/h3>\n\n\n
\u8ba1\u7b97\u5206\u7c7b\u5668\u7684\u6295\u7968\u6743\u91cd<\/h3>\n\n\n
\n
<\/li>\n<\/ol>\n\n\n\n
<\/li>\n<\/ol>\n\n\n\n
\n
<\/li>\n
<\/li>\n
<\/li>\n
<\/li>\n
<\/li>\n<\/ul>\n\n\n\u89e3\u91ca<\/h3>\n\n\n