6.1 Evaluation of an online technology leadership master's program  (Page 9/16)

The SPSS software was used to conduct a Cronbach’s α (alpha) test to assess internal consistency of the Likert-style items. Generally, Cronbach's alpha will increase as inter-correlations between test items increases, so it is often termed an internal consistency predictor of test score reliability (Leech, Barrett,&Morgan, 2005). The Cronbach’s alpha for the 5 questions related to Web 2.0 tools was excellent (α = 0.908). The 7 questions related to online teaching strategies appeared to be good (α = 0.848). The 6 questions about electronic portfolios were poor (α = 0.590).

As a result of the Cronbach’s alpha results, the electronic portfolio items were modified, leadership and technology professional development items were added, and the survey was again piloted. The subsequent pilot survey was administered to a different set of persons ( N =25) representative of the Educational Technology Leadership Master’s program who served in technical assistant or program participant roles, but were not ETL graduates. Moreover, field experts which included university professors and education professionals, provided feedback regarding the instrument construction. Minor revisions were made to clarify the items. Then using SurveyMonkey™, the revised (third) online survey was electronically distributed to 271 ETL graduates. Respondents were permitted to respond anonymously to the Likert items and open-ended responses.

In addition to collecting quantitative and qualitative survey data from a pool of 271 ETL program graduates, 60 graduates’ electronic portfolios were purposefully selected and examined. The 60 electronic portfolios were representative of graduates who completed the program during the same timeframe as the 271 ETL graduate data pool. The graduates’ electronic portfolios were a collection of artifacts and reflections gathered throughout the program coursework. Graduates’ writing and electronic portfolio components were analyzed to obtain qualitative data regarding viewpoints and perceptions about online learning, Web 2.0 tools, technology professional development, and leadership. Specific sub-research questions were developed to guide an issue-focused analysis of the data. An issue-focused description generally moved from one discussion of the issue to that within another area, but indicated ways the issues were connected to each other (Weiss, 1994). The main qualitative focused issue was related to what graduates say about the effectiveness of the ETL master’s program to advance personal leadership in the use of online learning, Web 2.0 tools, and technology professional development and whether there was of evidence of transference into PK-12 schools. The 60 graduates’ electronic portfolios were coded related to leadership in the following three major categories: online learning, Web 2.0 tools, and technology professional development. Then the categories were sorted to reveal key elements associated with the graduates’ work sampling such as personal leadership roles, online learning, technology tools, and professional development. Lastly, the identified key qualitative elements were incorporated with the quantitative data, thereby indicating program outcome effectiveness.