2023-02-25

SQL Query Optimisation for multiple table jons with millions of records

Can anyone suggest me a way to optimise the given query? It has multiple joins and if we try it with larger dataset of 10M records. The query takes much amount of time.

SELECT 
    AUDM.distributeTS AS distributionDate,
    ASMT.name,
    ACM.studentid,
    AUDM.users AS distributedTo,
    ROUND(AUM.totaluser * 100 / AUDM.users) AS participation,
    ROUND(AUM.score * 5 / (AUM.totaluser * AQM.qi)) AS performance
FROM
    (SELECT 
        name, assessmentId
    FROM
        Assessment
    WHERE
        type IN ('PSYCHOMETRIC' , 'QUANTITATIVE', '')
            AND removed = FALSE) ASMT
        LEFT JOIN
    (SELECT 
        studentid, assessmentId
    FROM
        AssessmentCreatorMap) ACM ON ACM.assessmentId = ASMT.AssessmentId
        LEFT JOIN
    (SELECT 
        assessmentId, COUNT(assessmentId) AS qi
    FROM
        AssessmentQuestionMap
    GROUP BY assessmentId) AQM ON AQM.assessmentId = ASMT.assessmentId
        LEFT JOIN
    (SELECT 
         COUNT(userId) AS users, distributeTS, assessmentId
    FROM
        AssessmentUserDistributeMap 
    GROUP BY assessmentId) AUDM ON AUDM.assessmentId = ASMT.assessmentId
        LEFT JOIN
    (SELECT 
        assessmentId,
            COUNT(assessmentId) AS totaluser,
            SUM(assessmentScore) AS score
    FROM
        AssessmentUserMap
    JOIN Student ON AssessmentUserMap.studentId = Student.studentid
    WHERE
        enrollmentDate IS NOT NULL
            AND isDeleted = FALSE
    GROUP BY assessmentId) AUM ON AUM.assessmentId = ASMT.assessmentId
ORDER BY ASMT.AssessmentId DESC
LIMIT 0 , 15;

explain yields the following result.

'1', 'PRIMARY', 'Assessment', NULL, 'index', NULL, 'PRIMARY', '4', NULL, '1', '5.00', 'Using where; Backward index scan'
'1', 'PRIMARY', 'AssessmentCreatorMap', NULL, 'ref', 'fk_AssessmentCreatorMap_aid_idx', 'fk_AssessmentCreatorMap_aid_idx', '5', 'OustMe_UAT.Assessment.AssessmentId', '1', '100.00', NULL
'1', 'PRIMARY', '<derived4>', NULL, 'ref', '<auto_key0>', '<auto_key0>', '5', 'OustMe_UAT.Assessment.AssessmentId', '10', '100.00', NULL
'1', 'PRIMARY', '<derived5>', NULL, 'ref', '<auto_key0>', '<auto_key0>', '5', 'OustMe_UAT.Assessment.AssessmentId', '601', '100.00', NULL
'1', 'PRIMARY', '<derived6>', NULL, 'ref', '<auto_key0>', '<auto_key0>', '5', 'OustMe_UAT.Assessment.AssessmentId', '10', '100.00', NULL
'6', 'DERIVED', 'AssessmentUserMap', NULL, 'ALL', 'fk_AssessmentUserMap_assessmentid_idx,fk_aum_studentid,idx_AssessmentUserMap_assessmentId_enrollmentDate,idx_AssessmentUserMap_assessmentId_studentid', NULL, NULL, NULL, '1055', '90.00', 'Using where; Using temporary'
'6', 'DERIVED', 'Student', NULL, 'eq_ref', 'studentid_UNIQUE,idx_Student_studentid,fk_student_isdel', 'studentid_UNIQUE', '182', 'OustMe_UAT.AssessmentUserMap.studentid', '1', '50.00', 'Using index condition; Using where'
'5', 'DERIVED', 'AssessmentUserDistributeMap', NULL, 'index', 'fk_AssessmentUserDistributeMap_aid_idx,idx_AssessmentUserDistributeMap_assessmentId_userId,idx_assessmentUserDistributeMap_userId_assessmentId', 'fk_AssessmentUserDistributeMap_aid_idx', '5', NULL, '397282', '100.00', NULL
'4', 'DERIVED', 'AssessmentQuestionMap', NULL, 'index', 'fk_AssessmentQuestionMap_aid_idx', 'fk_AssessmentQuestionMap_aid_idx', '5', NULL, '3308', '100.00', 'Using index'

most of the tables have the indexes already . Please comment if there is any need to add a new index or how can we rewrite the query to produce the same resultset.



No comments:

Post a Comment