Semi-Confirmatory Factor Analysis

Po-Hsien Huang

In this example, we will show how to use lslx to conduct semi-confirmatory factor analysis. The example uses data HolzingerSwineford1939 in the package lavaan. Hence, lavaan must be installed.

Model Specification

In the following specification, x1 - x9 is assumed to be measurements of 3 latent factors: visual, textual, and speed.

model_fa <- "visual  :=> x1 + x2 + x3
             textual :=> x4 + x5 + x6
             speed   :=> x7 + x8 + x9
             visual  :~> x4 + x5 + x6 + x7 + x8 + x9
             textual :~> x1 + x2 + x3 + x7 + x8 + x9
             speed   :~> x1 + x2 + x3 + x4 + x5 + x6
             visual  <=> fix(1)* visual
             textual <=> fix(1)* textual
             speed   <=> fix(1)* speed"

The operator :=> means that the LHS latent factors is defined by the RHS observed variables. In particular, the loadings are freely estimated. The operator :~> also means that the LHS latent factors is defined by the RHS observed variables, but these loadings are set as penalized coefficients. In this model, visual is mainly measured by x1 - x3, textual is mainly measured by x4 - x6, and speed is mainly measured by x7 - x9. However, the inclusion of the penalized loadings indicates that each measurement may not be only influenced by one latent factor. The operator <=> means that the LHS and RHS variables/factors are covaried. If the LHS and RHS variable/factor are the same, <=> specifies the variance of that variable/factor. For scale setting, visual <=> fix(1) * visual makes the variance of visual to be zero. Details of model syntax can be found in the section of Model Syntax via ?lslx.

Object Initialization

lslx is written as an R6 class. Every time we conduct analysis with lslx, an lslx object must be initialized. The following code initializes an lslx object named lslx_fa.

library(lslx)
lslx_fa <- lslx$new(model = model_fa, data = lavaan::HolzingerSwineford1939)
An 'lslx' R6 class is initialized via 'data' argument. 
  Response Variables: x1 x2 x3 x4 x5 x6 x7 x8 x9 
  Latent Factors: visual textual speed 

Here, lslx is the object generator for lslx object and $new() is the build-in method of lslx to generate a new lslx object. The initialization of lslx requires users to specify a model for model specification (argument model) and a data to be fitted (argument sample_data). The data set must contain all the observed variables specified in the given model. In is also possible to initialize an lslx object via sample moments (see vignette("structural-equation-modeling")).

To see the model specification, we may use the $extract_specification() method.

lslx_fa$extract_specification()
                             relation    left   right group reference matrix block  type start
x1<-1/g                         x1<-1      x1       1     g     FALSE  alpha  y<-1  free    NA
x2<-1/g                         x2<-1      x2       1     g     FALSE  alpha  y<-1  free    NA
x3<-1/g                         x3<-1      x3       1     g     FALSE  alpha  y<-1  free    NA
x4<-1/g                         x4<-1      x4       1     g     FALSE  alpha  y<-1  free    NA
x5<-1/g                         x5<-1      x5       1     g     FALSE  alpha  y<-1  free    NA
x6<-1/g                         x6<-1      x6       1     g     FALSE  alpha  y<-1  free    NA
x7<-1/g                         x7<-1      x7       1     g     FALSE  alpha  y<-1  free    NA
x8<-1/g                         x8<-1      x8       1     g     FALSE  alpha  y<-1  free    NA
x9<-1/g                         x9<-1      x9       1     g     FALSE  alpha  y<-1  free    NA
x1<-visual/g               x1<-visual      x1  visual     g     FALSE   beta  y<-f  free    NA
x2<-visual/g               x2<-visual      x2  visual     g     FALSE   beta  y<-f  free    NA
x3<-visual/g               x3<-visual      x3  visual     g     FALSE   beta  y<-f  free    NA
x4<-visual/g               x4<-visual      x4  visual     g     FALSE   beta  y<-f   pen    NA
x5<-visual/g               x5<-visual      x5  visual     g     FALSE   beta  y<-f   pen    NA
x6<-visual/g               x6<-visual      x6  visual     g     FALSE   beta  y<-f   pen    NA
x7<-visual/g               x7<-visual      x7  visual     g     FALSE   beta  y<-f   pen    NA
x8<-visual/g               x8<-visual      x8  visual     g     FALSE   beta  y<-f   pen    NA
x9<-visual/g               x9<-visual      x9  visual     g     FALSE   beta  y<-f   pen    NA
x1<-textual/g             x1<-textual      x1 textual     g     FALSE   beta  y<-f   pen    NA
x2<-textual/g             x2<-textual      x2 textual     g     FALSE   beta  y<-f   pen    NA
x3<-textual/g             x3<-textual      x3 textual     g     FALSE   beta  y<-f   pen    NA
x4<-textual/g             x4<-textual      x4 textual     g     FALSE   beta  y<-f  free    NA
x5<-textual/g             x5<-textual      x5 textual     g     FALSE   beta  y<-f  free    NA
x6<-textual/g             x6<-textual      x6 textual     g     FALSE   beta  y<-f  free    NA
x7<-textual/g             x7<-textual      x7 textual     g     FALSE   beta  y<-f   pen    NA
x8<-textual/g             x8<-textual      x8 textual     g     FALSE   beta  y<-f   pen    NA
x9<-textual/g             x9<-textual      x9 textual     g     FALSE   beta  y<-f   pen    NA
x1<-speed/g                 x1<-speed      x1   speed     g     FALSE   beta  y<-f   pen    NA
x2<-speed/g                 x2<-speed      x2   speed     g     FALSE   beta  y<-f   pen    NA
x3<-speed/g                 x3<-speed      x3   speed     g     FALSE   beta  y<-f   pen    NA
x4<-speed/g                 x4<-speed      x4   speed     g     FALSE   beta  y<-f   pen    NA
x5<-speed/g                 x5<-speed      x5   speed     g     FALSE   beta  y<-f   pen    NA
x6<-speed/g                 x6<-speed      x6   speed     g     FALSE   beta  y<-f   pen    NA
x7<-speed/g                 x7<-speed      x7   speed     g     FALSE   beta  y<-f  free    NA
x8<-speed/g                 x8<-speed      x8   speed     g     FALSE   beta  y<-f  free    NA
x9<-speed/g                 x9<-speed      x9   speed     g     FALSE   beta  y<-f  free    NA
visual<->visual/g     visual<->visual  visual  visual     g     FALSE    phi f<->f fixed     1
textual<->visual/g   textual<->visual textual  visual     g     FALSE    phi f<->f  free    NA
speed<->visual/g       speed<->visual   speed  visual     g     FALSE    phi f<->f  free    NA
textual<->textual/g textual<->textual textual textual     g     FALSE    phi f<->f fixed     1
speed<->textual/g     speed<->textual   speed textual     g     FALSE    phi f<->f  free    NA
speed<->speed/g         speed<->speed   speed   speed     g     FALSE    phi f<->f fixed     1
x1<->x1/g                     x1<->x1      x1      x1     g     FALSE    phi y<->y  free    NA
x2<->x2/g                     x2<->x2      x2      x2     g     FALSE    phi y<->y  free    NA
x3<->x3/g                     x3<->x3      x3      x3     g     FALSE    phi y<->y  free    NA
x4<->x4/g                     x4<->x4      x4      x4     g     FALSE    phi y<->y  free    NA
x5<->x5/g                     x5<->x5      x5      x5     g     FALSE    phi y<->y  free    NA
x6<->x6/g                     x6<->x6      x6      x6     g     FALSE    phi y<->y  free    NA
x7<->x7/g                     x7<->x7      x7      x7     g     FALSE    phi y<->y  free    NA
x8<->x8/g                     x8<->x8      x8      x8     g     FALSE    phi y<->y  free    NA
x9<->x9/g                     x9<->x9      x9      x9     g     FALSE    phi y<->y  free    NA
                    label
x1<-1/g              <NA>
x2<-1/g              <NA>
x3<-1/g              <NA>
x4<-1/g              <NA>
x5<-1/g              <NA>
x6<-1/g              <NA>
x7<-1/g              <NA>
x8<-1/g              <NA>
x9<-1/g              <NA>
x1<-visual/g         <NA>
x2<-visual/g         <NA>
x3<-visual/g         <NA>
x4<-visual/g         <NA>
x5<-visual/g         <NA>
x6<-visual/g         <NA>
x7<-visual/g         <NA>
x8<-visual/g         <NA>
x9<-visual/g         <NA>
x1<-textual/g        <NA>
x2<-textual/g        <NA>
x3<-textual/g        <NA>
x4<-textual/g        <NA>
x5<-textual/g        <NA>
x6<-textual/g        <NA>
x7<-textual/g        <NA>
x8<-textual/g        <NA>
x9<-textual/g        <NA>
x1<-speed/g          <NA>
x2<-speed/g          <NA>
x3<-speed/g          <NA>
x4<-speed/g          <NA>
x5<-speed/g          <NA>
x6<-speed/g          <NA>
x7<-speed/g          <NA>
x8<-speed/g          <NA>
x9<-speed/g          <NA>
visual<->visual/g    <NA>
textual<->visual/g   <NA>
speed<->visual/g     <NA>
textual<->textual/g  <NA>
speed<->textual/g    <NA>
speed<->speed/g      <NA>
x1<->x1/g            <NA>
x2<->x2/g            <NA>
x3<->x3/g            <NA>
x4<->x4/g            <NA>
x5<->x5/g            <NA>
x6<->x6/g            <NA>
x7<->x7/g            <NA>
x8<->x8/g            <NA>
x9<->x9/g            <NA>

The row names show the coefficient names. The most two relevant columns are type which shows the type of the coefficient and start which gives the starting value. In lslx, many extract related methods are defined. extract related methods can be used to extract quantities stored in lslx object. For details, please see the section of Extract-Related Methods via ?lslx.

Model Fitting

After an lslx object is initialized, method $fit() can be used to fit the specified model to the given data.

lslx_fa$fit(penalty_method = "mcp", 
            lambda_grid = seq(.01, .60, .01), 
            delta_grid = c(1.5, 3.0, Inf))
CONGRATS: Algorithm converges under EVERY specified penalty level.
  Specified Tolerance for Convergence: 0.001 
  Specified Maximal Number of Iterations: 100 

The fitting process requires users to specify the penalty method (argument penalty_method) and the considered penalty levels (argument lambda_grid and delta_grid). In this example, the mcp penalty is implemented on the lambda grid seq(.01, .60, .01) and delta grid c(1.5, 3, Inf). Note that in this example lambda = 0 is not considered because it may result in unidentified model. All the fitting result will be stored in the fitting field of lslx_fa.

Model Summarizing

Unlike traditional SEM analysis, lslx fits the model into data under all the penalty levels considered. To summarize the fitting result, a selector to determine an optimal penalty level must be specified. Available selectors can be found in the section of Penalty Level Selection via ?lslx. The following code summarize the fitting result under the penalty level selected by Bayesian information criterion (BIC).

lslx_fa$summarize(selector = "bic")
General Information                                                            
   number of observations                                301
   number of complete observations                       301
   number of missing patterns                           none
   number of groups                                        1
   number of responses                                     9
   number of factors                                       3
   number of free coefficients                            30
   number of penalized coefficients                       18

Numerical Conditions                                                            
   selected lambda                                     0.140
   selected delta                                      1.500
   selected step                                        none
   objective value                                     0.158
   objective gradient absolute maximum                 0.000
   objective Hessian convexity                         0.593
   number of iterations                                4.000
   loss value                                          0.103
   number of non-zero coefficients                    34.000
   degrees of freedom                                 20.000
   robust degrees of freedom                          20.640
   scaling factor                                      1.032

Fit Indices                                                            
   root mean square error of approximation (rmsea)     0.043
   comparative fit index (cfi)                         0.988
   non-normed fit index (nnfi)                         0.978
   standardized root mean of residual (srmr)           0.030

Likelihood Ratio Test
                    statistic         df    p-value
   unadjusted          30.919     20.000      0.056
   mean-adjusted       29.961     20.000      0.070

Root Mean Square Error of Approximation Test
                     estimate      lower      upper
   unadjusted           0.043      0.000      0.075
   mean-adjusted        0.041      0.000      0.075

Coefficient Test (Std.Error = "sandwich")
  Factor Loading
                      type  estimate  std.error  z-value  p-value  lower  upper
       x1<-visual     free     0.709      0.095    7.482    0.000  0.523  0.895
       x2<-visual     free     0.555      0.082    6.799    0.000  0.395  0.715
       x3<-visual     free     0.744      0.070   10.577    0.000  0.606  0.882
       x4<-visual      pen     0.000        -        -        -      -      -  
       x5<-visual      pen    -0.102      0.073   -1.401    0.081 -0.244  0.041
       x6<-visual      pen     0.000        -        -        -      -      -  
       x7<-visual      pen    -0.255      0.119   -2.151    0.016 -0.488 -0.023
       x8<-visual      pen     0.000        -        -        -      -      -  
       x9<-visual      pen     0.323      0.075    4.287    0.000  0.175  0.471
      x1<-textual      pen     0.255      0.081    3.136    0.001  0.096  0.415
      x2<-textual      pen     0.000        -        -        -      -      -  
      x3<-textual      pen     0.000        -        -        -      -      -  
      x4<-textual     free     0.987      0.061   16.133    0.000  0.867  1.106
      x5<-textual     free     1.143      0.061   18.582    0.000  1.022  1.263
      x6<-textual     free     0.913      0.058   15.727    0.000  0.799  1.027
      x7<-textual      pen     0.000        -        -        -      -      -  
      x8<-textual      pen     0.000        -        -        -      -      -  
      x9<-textual      pen     0.000        -        -        -      -      -  
        x1<-speed      pen     0.000        -        -        -      -      -  
        x2<-speed      pen     0.000        -        -        -      -      -  
        x3<-speed      pen     0.000        -        -        -      -      -  
        x4<-speed      pen     0.000        -        -        -      -      -  
        x5<-speed      pen     0.000        -        -        -      -      -  
        x6<-speed      pen     0.000        -        -        -      -      -  
        x7<-speed     free     0.825      0.106    7.805    0.000  0.618  1.032
        x8<-speed     free     0.731      0.069   10.604    0.000  0.596  0.866
        x9<-speed     free     0.499      0.063    7.962    0.000  0.376  0.622

  Covariance
                      type  estimate  std.error  z-value  p-value  lower  upper
 textual<->visual     free     0.325      0.086    3.765    0.000  0.156  0.494
   speed<->visual     free     0.375      0.100    3.731    0.000  0.178  0.571
  speed<->textual     free     0.278      0.078    3.572    0.000  0.126  0.431

  Variance
                      type  estimate  std.error  z-value  p-value  lower  upper
  visual<->visual    fixed     1.000        -        -        -      -      -  
textual<->textual    fixed     1.000        -        -        -      -      -  
    speed<->speed    fixed     1.000        -        -        -      -      -  
          x1<->x1     free     0.671      0.113    5.951    0.000  0.450  0.891
          x2<->x2     free     1.073      0.106   10.165    0.000  0.866  1.280
          x3<->x3     free     0.720      0.090    7.955    0.000  0.542  0.897
          x4<->x4     free     0.377      0.050    7.504    0.000  0.279  0.475
          x5<->x5     free     0.416      0.060    6.875    0.000  0.297  0.534
          x6<->x6     free     0.362      0.046    7.902    0.000  0.273  0.452
          x7<->x7     free     0.595      0.109    5.444    0.000  0.381  0.809
          x8<->x8     free     0.488      0.084    5.780    0.000  0.323  0.654
          x9<->x9     free     0.540      0.064    8.431    0.000  0.414  0.666

  Intercept
                      type  estimate  std.error  z-value  p-value  lower  upper
            x1<-1     free     4.936      0.067   73.473    0.000  4.804  5.067
            x2<-1     free     6.088      0.068   89.855    0.000  5.955  6.221
            x3<-1     free     2.250      0.065   34.579    0.000  2.123  2.378
            x4<-1     free     3.061      0.067   45.694    0.000  2.930  3.192
            x5<-1     free     4.341      0.074   58.452    0.000  4.195  4.486
            x6<-1     free     2.186      0.063   34.667    0.000  2.062  2.309
            x7<-1     free     4.186      0.063   66.766    0.000  4.063  4.309
            x8<-1     free     5.527      0.058   94.854    0.000  5.413  5.641
            x9<-1     free     5.374      0.058   92.546    0.000  5.260  5.488

In this example, we can see that most penalized coefficients are estimated as zero under the selected penalty level except for x9<-visual, which shows the benefit of using the semi-confirmatory approach. The summarize method also shows the result of significance tests for the coefficients. In lslx, the default standard errors are calculated based on sandwich formula whenever raw data is available. It is generally valid even when the model is misspecified and the data is not normal. However, it may not be valid after selecting an optimal penalty level.

Visualization

lslx provides four methods for visualizing the fitting results. The method $plot_numerical_condition() shows the numerical condition under all the penalty levels. The following code plots the values of n_iter_out (number of iterations in outer loop), objective_gradient_abs_max (maximum of absolute value of gradient of objective function), and objective_hessian_convexity (minimum of univariate approximate hessian). The plot can be used to evaluate the quality of numerical optimization.

lslx_fa$plot_numerical_condition()

The method $plot_information_criterion() shows the values of information criteria under all the penalty levels.

lslx_fa$plot_information_criterion()

The method $plot_fit_index() shows the values of fit indices under all the penalty levels.

lslx_fa$plot_fit_index()

The method $plot_coefficient() shows the solution path of coefficients in the given block. The following code plots the solution paths of all coefficients in the block y<-f, which contains all the regression coefficients from latent factors to observed variables (i.e., factor loadings).

lslx_fa$plot_coefficient(block = "y<-f")

Objects Extraction

In lslx, many quantities related to SEM can be extracted by extract-related method. For example, the loading matrix can be obtained by

lslx_fa$extract_coefficient_matrix(selector = "bic", block = "y<-f")
$g
   visual textual speed
x1  0.709   0.255 0.000
x2  0.555   0.000 0.000
x3  0.744   0.000 0.000
x4  0.000   0.987 0.000
x5 -0.102   1.143 0.000
x6  0.000   0.913 0.000
x7 -0.255   0.000 0.825
x8  0.000   0.000 0.731
x9  0.323   0.000 0.499

The model-implied covariance matrix and residual matrix can be obtained by

lslx_fa$extract_implied_cov(selector = "bic")
$g
       x1     x2     x3    x4    x5    x6     x7    x8    x9
x1 1.3560 0.4395 0.5890 0.479 0.474 0.443 0.0754 0.246 0.424
x2 0.4395 1.3807 0.4128 0.178 0.150 0.165 0.0297 0.152 0.283
x3 0.5890 0.4128 1.2729 0.238 0.200 0.221 0.0398 0.204 0.379
x4 0.4791 0.1778 0.2384 1.350 1.095 0.901 0.1446 0.200 0.240
x5 0.4744 0.1496 0.2005 1.095 1.657 1.013 0.1620 0.204 0.227
x6 0.4434 0.1646 0.2206 0.901 1.013 1.196 0.1338 0.186 0.223
x7 0.0754 0.0297 0.0398 0.145 0.162 0.134 1.1830 0.533 0.381
x8 0.2459 0.1518 0.2035 0.200 0.204 0.186 0.5328 1.022 0.453
x9 0.4237 0.2829 0.3792 0.240 0.227 0.223 0.3813 0.453 1.014
lslx_fa$extract_residual_cov(selector = "bic")
$g
         x1       x2       x3        x4       x5        x6        x7        x8       x9
x1 -0.00241  0.03208  0.00914 -0.025731  0.03381 -0.011432 -0.009379 -0.017971 -0.03465
x2  0.03208 -0.00113 -0.03827 -0.031094 -0.06154 -0.082977  0.126476  0.042179  0.03892
x3  0.00914 -0.03827 -0.00201  0.030192  0.08817 -0.023522 -0.048498 -0.008799  0.00539
x4 -0.02573 -0.03109  0.03019 -0.000262 -0.00285  0.005299 -0.075190  0.074893 -0.00292
x5  0.03381 -0.06154  0.08817 -0.002849 -0.00320 -0.001270  0.018988  0.023798 -0.06857
x6 -0.01143 -0.08298 -0.02352  0.005299 -0.00127 -0.000288 -0.010305  0.020105 -0.01348
x7 -0.00938  0.12648 -0.04850 -0.075190  0.01899 -0.010305 -0.000114 -0.002464  0.00798
x8 -0.01797  0.04218 -0.00880  0.074893  0.02380  0.020105 -0.002464 -0.000243 -0.00436
x9 -0.03465  0.03892  0.00539 -0.002918 -0.06857 -0.013483  0.007976 -0.004364 -0.00094

Wrapper Function plsem() and S3 Methods

After version 0.6.3, the previous analysis can be equivalently conducted by plsem() with lavaan style model syntax.

model_fa <- "visual  =~ x1 + x2 + x3
             textual =~ x4 + x5 + x6
             speed   =~ x7 + x8 + x9
             pen() * visual  =~ x4 + x5 + x6 + x7 + x8 + x9
             pen() * textual =~ x1 + x2 + x3 + x7 + x8 + x9
             pen() * speed   =~ x1 + x2 + x3 + x4 + x5 + x6
             visual  ~~ 1 * visual
             textual ~~ 1 * textual
             speed   ~~ 1 * speed"
             
lslx_fa <- plsem(model = model_fa, 
                 data = lavaan::HolzingerSwineford1939,
                 penalty_method = "mcp", 
                 lambda_grid = seq(.01, .60, .01), 
                 delta_grid = c(1.5, 3.0, Inf))
An 'lslx' R6 class is initialized via 'data' argument. 
  Response Variables: x1 x2 x3 x4 x5 x6 x7 x8 x9 
  Latent Factors: visual textual speed 
CONGRATS: Algorithm converges under EVERY specified penalty level.
  Specified Tolerance for Convergence: 0.001 
  Specified Maximal Number of Iterations: 100 
summary(lslx_fa, selector = "bic")
General Information                                                            
   number of observations                                301
   number of complete observations                       301
   number of missing patterns                           none
   number of groups                                        1
   number of responses                                     9
   number of factors                                       3
   number of free coefficients                            30
   number of penalized coefficients                       18

Numerical Conditions                                                            
   selected lambda                                     0.140
   selected delta                                      1.500
   selected step                                        none
   objective value                                     0.158
   objective gradient absolute maximum                 0.000
   objective Hessian convexity                         0.593
   number of iterations                                4.000
   loss value                                          0.103
   number of non-zero coefficients                    34.000
   degrees of freedom                                 20.000
   robust degrees of freedom                          20.640
   scaling factor                                      1.032

Fit Indices                                                            
   root mean square error of approximation (rmsea)     0.043
   comparative fit index (cfi)                         0.988
   non-normed fit index (nnfi)                         0.978
   standardized root mean of residual (srmr)           0.030

Likelihood Ratio Test
                    statistic         df    p-value
   unadjusted          30.919     20.000      0.056
   mean-adjusted       29.961     20.000      0.070

Root Mean Square Error of Approximation Test
                     estimate      lower      upper
   unadjusted           0.043      0.000      0.075
   mean-adjusted        0.041      0.000      0.075

Coefficient Test (Std.Error = "sandwich")
  Factor Loading
                      type  estimate  std.error  z-value  p-value  lower  upper
       x1<-visual     free     0.709      0.095    7.482    0.000  0.523  0.895
       x2<-visual     free     0.555      0.082    6.799    0.000  0.395  0.715
       x3<-visual     free     0.744      0.070   10.577    0.000  0.606  0.882
       x4<-visual      pen     0.000        -        -        -      -      -  
       x5<-visual      pen    -0.102      0.073   -1.401    0.081 -0.244  0.041
       x6<-visual      pen     0.000        -        -        -      -      -  
       x7<-visual      pen    -0.255      0.119   -2.151    0.016 -0.488 -0.023
       x8<-visual      pen     0.000        -        -        -      -      -  
       x9<-visual      pen     0.323      0.075    4.287    0.000  0.175  0.471
      x1<-textual      pen     0.255      0.081    3.136    0.001  0.096  0.415
      x2<-textual      pen     0.000        -        -        -      -      -  
      x3<-textual      pen     0.000        -        -        -      -      -  
      x4<-textual     free     0.987      0.061   16.133    0.000  0.867  1.106
      x5<-textual     free     1.143      0.061   18.582    0.000  1.022  1.263
      x6<-textual     free     0.913      0.058   15.727    0.000  0.799  1.027
      x7<-textual      pen     0.000        -        -        -      -      -  
      x8<-textual      pen     0.000        -        -        -      -      -  
      x9<-textual      pen     0.000        -        -        -      -      -  
        x1<-speed      pen     0.000        -        -        -      -      -  
        x2<-speed      pen     0.000        -        -        -      -      -  
        x3<-speed      pen     0.000        -        -        -      -      -  
        x4<-speed      pen     0.000        -        -        -      -      -  
        x5<-speed      pen     0.000        -        -        -      -      -  
        x6<-speed      pen     0.000        -        -        -      -      -  
        x7<-speed     free     0.825      0.106    7.805    0.000  0.618  1.032
        x8<-speed     free     0.731      0.069   10.604    0.000  0.596  0.866
        x9<-speed     free     0.499      0.063    7.962    0.000  0.376  0.622

  Covariance
                      type  estimate  std.error  z-value  p-value  lower  upper
 textual<->visual     free     0.325      0.086    3.765    0.000  0.156  0.494
   speed<->visual     free     0.375      0.100    3.731    0.000  0.178  0.571
  speed<->textual     free     0.278      0.078    3.572    0.000  0.126  0.431

  Variance
                      type  estimate  std.error  z-value  p-value  lower  upper
  visual<->visual    fixed     1.000        -        -        -      -      -  
textual<->textual    fixed     1.000        -        -        -      -      -  
    speed<->speed    fixed     1.000        -        -        -      -      -  
          x1<->x1     free     0.671      0.113    5.951    0.000  0.450  0.891
          x2<->x2     free     1.073      0.106   10.165    0.000  0.866  1.280
          x3<->x3     free     0.720      0.090    7.955    0.000  0.542  0.897
          x4<->x4     free     0.377      0.050    7.504    0.000  0.279  0.475
          x5<->x5     free     0.416      0.060    6.875    0.000  0.297  0.534
          x6<->x6     free     0.362      0.046    7.902    0.000  0.273  0.452
          x7<->x7     free     0.595      0.109    5.444    0.000  0.381  0.809
          x8<->x8     free     0.488      0.084    5.780    0.000  0.323  0.654
          x9<->x9     free     0.540      0.064    8.431    0.000  0.414  0.666

  Intercept
                      type  estimate  std.error  z-value  p-value  lower  upper
            x1<-1     free     4.936      0.067   73.473    0.000  4.804  5.067
            x2<-1     free     6.088      0.068   89.855    0.000  5.955  6.221
            x3<-1     free     2.250      0.065   34.579    0.000  2.123  2.378
            x4<-1     free     3.061      0.067   45.694    0.000  2.930  3.192
            x5<-1     free     4.341      0.074   58.452    0.000  4.195  4.486
            x6<-1     free     2.186      0.063   34.667    0.000  2.062  2.309
            x7<-1     free     4.186      0.063   66.766    0.000  4.063  4.309
            x8<-1     free     5.527      0.058   94.854    0.000  5.413  5.641
            x9<-1     free     5.374      0.058   92.546    0.000  5.260  5.488